Re: [R] ROC within SEM

[Eric Berger]

Hi Ray,
Have you done any search at all? I did a search on "R package ROC latent
variable" and got several hits.
In particular the package randomLCA seems relevant

https://cran.r-project.org/web/packages/randomLCA/vignettes/randomLCA-package.pdf

I glanced at the documentation which mentions 2 other R packages that also
seem to address your needs - poLCA and BayesLCA.

HTH,
Eric


On Wed, Jun 6, 2018 at 11:48 PM, David Winsemius 
wrote:

>
> > On Jun 6, 2018, at 10:37 AM, Palmer, Raymond F 
> wrote:
> >
> > Dear R group.
> > Does anyone have an idea how to utilize a latent variable in an ROC
> (AUC) analysis? I want to create a latent variable, then use that latent
> construct as a continuous variable in an ROC.
> > I understand both SEM and ROC analysis can be done in R, but how to use
> the latent variable in the ROC is the issue.
> > Thanks for any insights.
> > Best, Ray
> >
> > Ray Palmer, Ph.D.
> > Professor
> > Department of Family and Community Medicine
> > University of Texas Health Science Center San Antonio
> > palm...@uthscsa.edu
> > 210-827-7681
> >
> >
> >   [[alternative HTML version deleted]]
>
> There are two sections from the Posting Guide that may be relevant:
>
> ---begin--
> Questions about statistics: The R mailing lists are primarily intended for
> questions and discussion about the R software. However, questions about
> statistical methodology are sometimes posted. If the question is well-asked
> and of interest to someone on the list, it may elicit an informative
> up-to-date answer. See also the Usenet groups sci.stat.consult (applied
> statistics and consulting) and sci.stat.math (mathematical stat and
> probability).
> ---end-
>
> Frankly I would not considered this question to be "well-asked" but anyone
> is free to dispute this or to take on the task of asking clarifying
> questions.
>
> A modern update might add to the Posting Guide in these more web-centric
> times, the CrossValidated.com site. https://stats.stackexchange.
> com/search?q=latent+variable+roc
>
> (Those newsgroups are basically defunct.)
>
> >
> ---begin--
> • No HTML posting (harder to detect spam) (note that this is the
> default in some mail clients - you may have to turn it off). Note that
> chances have become relatively high for ‘HTMLified’ e-mails to be
> completely intercepted (without notice to the sender).
> ---end-
>
> One of the effects of redistribution by a mail-server is the removal of
> the identity of your mail-client, so advice about that concern can only be
> "RTFM".
>
>
> > __
> > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/
> posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>
> David Winsemius
> Alameda, CA, USA
>
> 'Any technology distinguishable from magic is insufficiently advanced.'
>  -Gehm's Corollary to Clarke's Third Law
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/
> posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC within SEM

[David Winsemius]

> On Jun 6, 2018, at 10:37 AM, Palmer, Raymond F  wrote:
> 
> Dear R group.
> Does anyone have an idea how to utilize a latent variable in an ROC (AUC) 
> analysis? I want to create a latent variable, then use that latent construct 
> as a continuous variable in an ROC.
> I understand both SEM and ROC analysis can be done in R, but how to use the 
> latent variable in the ROC is the issue.
> Thanks for any insights.
> Best, Ray
> 
> Ray Palmer, Ph.D.
> Professor
> Department of Family and Community Medicine
> University of Texas Health Science Center San Antonio
> palm...@uthscsa.edu
> 210-827-7681
> 
> 
>   [[alternative HTML version deleted]]

There are two sections from the Posting Guide that may be relevant:

---begin--
Questions about statistics: The R mailing lists are primarily intended for 
questions and discussion about the R software. However, questions about 
statistical methodology are sometimes posted. If the question is well-asked and 
of interest to someone on the list, it may elicit an informative up-to-date 
answer. See also the Usenet groups sci.stat.consult (applied statistics and 
consulting) and sci.stat.math (mathematical stat and probability).
---end-

Frankly I would not considered this question to be "well-asked" but anyone is 
free to dispute this or to take on the task of asking clarifying questions. 

A modern update might add to the Posting Guide in these more web-centric times, 
the CrossValidated.com site. 
https://stats.stackexchange.com/search?q=latent+variable+roc

(Those newsgroups are basically defunct.)

> 
---begin--
• No HTML posting (harder to detect spam) (note that this is the 
default in some mail clients - you may have to turn it off). Note that chances 
have become relatively high for ‘HTMLified’ e-mails to be completely 
intercepted (without notice to the sender).
---end-

One of the effects of redistribution by a mail-server is the removal of the 
identity of your mail-client, so advice about that concern can only be "RTFM".


> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

David Winsemius
Alameda, CA, USA

'Any technology distinguishable from magic is insufficiently advanced.'   
-Gehm's Corollary to Clarke's Third Law

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] ROC within SEM

[Palmer, Raymond F]

Dear R group.
Does anyone have an idea how to utilize a latent variable in an ROC (AUC) 
analysis? I want to create a latent variable, then use that latent construct as 
a continuous variable in an ROC.
I understand both SEM and ROC analysis can be done in R, but how to use the 
latent variable in the ROC is the issue.
Thanks for any insights.
Best, Ray

Ray Palmer, Ph.D.
Professor
Department of Family and Community Medicine
University of Texas Health Science Center San Antonio
palm...@uthscsa.edu
210-827-7681


[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC curve for each fold in one plot

[PIKAL Petr]

Hi

I did not see any response but what about

par(mfrow=c(2,2))

before plotting?

Cheers
Petr

> -Original Message-
> From: R-help [mailto:r-help-boun...@r-project.org] On Behalf Of Elahe chalabi
> via R-help
> Sent: Monday, October 16, 2017 2:26 PM
> To: R-help Mailing List <r-help@r-project.org>
> Subject: [R] ROC curve for each fold in one plot
>
> Hi all,
>
> I have tried a 5 fold cross validation using caret package with random forest
> method on iris dataset as example. Then I need ROC curve for each fold:
>
>
>   > set.seed(1)
>   > train_control <- trainControl(method="cv", number=5,savePredictions =
> TRUE,classProbs = TRUE)
>   > output <- train(Species~., data=iris, trControl=train_control, 
> method="rf")
>   > library(pROC)
>   > selectedIndices <- output$pred$Resample == "Fold1"
>   >
> plot.roc(output$pred$obs[selectedIndices],output$pred$setosa[selectedIndices
> ])  > selectedIndices <- output$pred$Resample == "Fold2"
>   >
> plot.roc(output$pred$obs[selectedIndices],output$pred$setosa[selectedIndices
> ])
>   > selectedIndices <- output$pred$Resample == "Fold3"
>   >
> plot.roc(output$pred$obs[selectedIndices],output$pred$setosa[selectedIndices
> ])
>
> and the same for Fold4 and Fold5,now how can I bring all the plots in one plot
> with labels for each fold?
>
> Thanks for any help!
> Elahe
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.


Tento e-mail a jakékoliv k němu připojené dokumenty jsou důvěrné a jsou určeny 
pouze jeho adresátům.
Jestliže jste obdržel(a) tento e-mail omylem, informujte laskavě neprodleně 
jeho odesílatele. Obsah tohoto emailu i s přílohami a jeho kopie vymažte ze 
svého systému.
Nejste-li zamýšleným adresátem tohoto emailu, nejste oprávněni tento email 
jakkoliv užívat, rozšiřovat, kopírovat či zveřejňovat.
Odesílatel e-mailu neodpovídá za eventuální škodu způsobenou modifikacemi či 
zpožděním přenosu e-mailu.

V případě, že je tento e-mail součástí obchodního jednání:
- vyhrazuje si odesílatel právo ukončit kdykoliv jednání o uzavření smlouvy, a 
to z jakéhokoliv důvodu i bez uvedení důvodu.
- a obsahuje-li nabídku, je adresát oprávněn nabídku bezodkladně přijmout; 
Odesílatel tohoto e-mailu (nabídky) vylučuje přijetí nabídky ze strany příjemce 
s dodatkem či odchylkou.
- trvá odesílatel na tom, že příslušná smlouva je uzavřena teprve výslovným 
dosažením shody na všech jejích náležitostech.
- odesílatel tohoto emailu informuje, že není oprávněn uzavírat za společnost 
žádné smlouvy s výjimkou případů, kdy k tomu byl písemně zmocněn nebo písemně 
pověřen a takové pověření nebo plná moc byly adresátovi tohoto emailu případně 
osobě, kterou adresát zastupuje, předloženy nebo jejich existence je adresátovi 
či osobě jím zastoupené známá.

This e-mail and any documents attached to it may be confidential and are 
intended only for its intended recipients.
If you received this e-mail by mistake, please immediately inform its sender. 
Delete the contents of this e-mail with all attachments and its copies from 
your system.
If you are not the intended recipient of this e-mail, you are not authorized to 
use, disseminate, copy or disclose this e-mail in any manner.
The sender of this e-mail shall not be liable for any possible damage caused by 
modifications of the e-mail or by delay with transfer of the email.

In case that this e-mail forms part of business dealings:
- the sender reserves the right to end negotiations about entering into a 
contract in any time, for any reason, and without stating any reasoning.
- if the e-mail contains an offer, the recipient is entitled to immediately 
accept such offer; The sender of this e-mail (offer) excludes any acceptance of 
the offer on the part of the recipient containing any amendment or variation.
- the sender insists on that the respective contract is concluded only upon an 
express mutual agreement on all its aspects.
- the sender of this e-mail informs that he/she is not authorized to enter into 
any contracts on behalf of the company except for cases in which he/she is 
expressly authorized to do so in writing, and such authorization or power of 
attorney is submitted to the recipient or the person represented by the 
recipient, or the existence of such authorization is known to the recipient of 
the person represented by the recipient.
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] ROC curve for each fold in one plot

[Elahe chalabi via R-help]

Hi all,

I have tried a 5 fold cross validation using caret package with random forest 
method on iris dataset as example. Then I need ROC curve for each fold:


  > set.seed(1)
  > train_control <- trainControl(method="cv", number=5,savePredictions = 
TRUE,classProbs = TRUE) 
  > output <- train(Species~., data=iris, trControl=train_control, method="rf")
  > library(pROC)  
  > selectedIndices <- output$pred$Resample == "Fold1"
  > 
plot.roc(output$pred$obs[selectedIndices],output$pred$setosa[selectedIndices])  
> selectedIndices <- output$pred$Resample == "Fold2"  
  > 
plot.roc(output$pred$obs[selectedIndices],output$pred$setosa[selectedIndices])
  > selectedIndices <- output$pred$Resample == "Fold3"
  > 
plot.roc(output$pred$obs[selectedIndices],output$pred$setosa[selectedIndices])

and the same for Fold4 and Fold5,now how can I bring all the plots in one plot 
with labels for each fold?

Thanks for any help!
Elahe

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] ROC Curves for Neural Networks

[Richard Beckett]

Trying to plot an ROC curve to determine effectiveness of a neural network 
model in R. 

using the neuralnet package I created nn1a: 
nn1a-neuralnet(warstns ~ aigenlz+ 
p2l+anoclinv2+flinstab+mill+minconl+fllgdppclz+lpoplz+floil+flmtnestz+c_peaceyears,
 data=na.omit(dfram1), hidden=4) 

then using the lroc function from the epicalc package I tried lroc(nn1a) but 
came up with the error message 

Error in `colnames-`(`*tmp*`, value = c(Non-diseased, Diseased)) : 
  length of 'dimnames' [2] not equal to array extent 

Does anyone know any ways to get ROC curves to work with neural network models? 
Any other packages, model manipulation, etc? 

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC Curve: negative AUC

[brunosm]

Thanks,

Do you know how to correct it?

Cheers,

Bruno



--
View this message in context: 
http://r.789695.n4.nabble.com/ROC-Curve-negative-AUC-tp4650469p4650530.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] ROC Curve: negative AUC

[brunosm]

Hi all,

does anyone know why the area under the curve (AUC) is negative?

I'm using ROC function with a logistic regression, package Epi.

First time it happens...

Thanks a lot!

Bruno



--
View this message in context: 
http://r.789695.n4.nabble.com/ROC-Curve-negative-AUC-tp4650469.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC Curve: negative AUC

[Wensui Liu]

wrong direction for ranking

On Thu, Nov 22, 2012 at 1:58 PM, brunosm brunos...@gmail.com wrote:

 the area under the curve (AUC) is negative?

 I'm using ROC function with a logistic regression, package Epi.

 First time it happens...

 Thanks a lot!

 Bruno




-- 
==
WenSui Liu
Credit Risk Manager, 53 Bancorp
wensui@53.com
513-295-4370
==

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC Curve: negative AUC

[Thomas Lumley]

On Fri, Nov 23, 2012 at 7:58 AM, brunosm brunos...@gmail.com wrote:

 Hi all,

 does anyone know why the area under the curve (AUC) is negative?

 I'm using ROC function with a logistic regression, package Epi.

 First time it happens...


Have you looked at the ROC curve?

That should tell you whether there's some strange non-convexity going on or
whether the variable is just being put into the calculations backwards.

   -thomas


-- 
Thomas Lumley
Professor of Biostatistics
University of Auckland

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC with more than one predictor

[Suzen, Mehmet]

On Tue, Oct 23, 2012 at 11:52 PM, Luigi marongiu.lu...@gmail.com wrote:

 Any tip on how to proceed?

You may want to do check rocplus package. Its vignette is pretty good.
http://cran.r-project.org/web/packages/rocplus/vignettes/rocplus.pdf
In your case, I think pairwise comparison would be one approach


Best,
Mehmet

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] ROC with more than one predictor

[Luigi]

Dear all,
I would like to perform a receiver operating characteristics (ROC) curve
analysis on data obtained using molecular assays. 

I have here an example: the first column is the sample number, the second is
the response variable and the last three columns are the predictors. I wish
to set the formula to test as R ~ P1 + P2 + P3 in order to obtain the
cut-offs for each of the predictors combined. 

Any tip on how to proceed? 

Thank you
Best regards,
Luigi 

R   P1  P2  P3
1   0   0.725196908 0   0
2   0   1.089876198 0   0
3   0   1.487365838 0   0
4   0   6.214688703 3.625   0
5   0   48.26356415 13.40
6   1   9.494694175 0   0
7   1   15.5773 4.666   0
8   1   45.53510205 15.57   0
9   1   0.373389928 12.55   1
10  1   0.692909116 0   1
11  1   15.05400384 3.451
12  1   20.83909961 6.9 1
13  1   23.24171357 0   1
14  1   43.51470584 0   1
15  1   49.80855616 3.551
16  1   664.0886249 10.37   1
17  2   60.85094536 3.725   0
18  2   62.7104185  0   1
19  2   143.7426283 6.5 1

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC curves with ROCR

[Jessica Streicher]

Wild guess :
You need to have some kind of score for ROC to work (so you can work with a 
threshold), so a predict() that only returns classlabels probably won't work.

On 13.07.2012, at 15:25, blerta wrote:

 Hi, 
 
 I don't really understand how ROCR works.  Here's another example with a
 randomforest model: I have the training dataset(bank_training) and testing
 dataset(bank_testing) and I ran a randomForest as below:
 
bankrf-randomForest(y~., bank_training, mtry=4, ntree=2,  
keep.forest=TRUE,importance=TRUE) 
bankrf.pred-predict(bankrf, bank_testing)
library(ROCR)
pred-prediction(bankrf.pred$y, bank_testing$y)
 
 Here I get the error that the prediction format is incorrect? Where is the
 mistake?
 
 Thanks in advance
 
 --
 View this message in context: 
 http://r.789695.n4.nabble.com/ROC-curves-with-ROCR-tp4636435.html
 Sent from the R help mailing list archive at Nabble.com.
 
 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.


[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] ROC curves with ROCR

[blerta]

Hi, 

I don't really understand how ROCR works.  Here's another example with a
randomforest model: I have the training dataset(bank_training) and testing
dataset(bank_testing) and I ran a randomForest as below:

bankrf-randomForest(y~., bank_training, mtry=4, ntree=2,  
keep.forest=TRUE,importance=TRUE) 
bankrf.pred-predict(bankrf, bank_testing)
library(ROCR)
pred-prediction(bankrf.pred$y, bank_testing$y)

Here I get the error that the prediction format is incorrect? Where is the
mistake?

Thanks in advance

--
View this message in context: 
http://r.789695.n4.nabble.com/ROC-curves-with-ROCR-tp4636435.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC Analysis

[Camille Leclerc]

Hi everybody,

Pascal, your code works, but when I want to do the graph I have an error
message. 

here is my code :
x-rev(unlist(pred@cutoffs))
tpf-unlist(performance(pred, tpr)@y.values)
fpf-unlist(performance(pred,fpr)@y.values)
ll-length(x)
p-(tpf[1:(ll-1)]-tpf[2:ll])/(fpf[1:(ll-1)]-fpf[2:ll])
plot(x,p)

*Erreur dans xy.coords(x, y, xlabel, ylabel, log) : 
'x' and 'y' lengths differ*

So, when I look the lenghts of x and p, I have this :
*x : numeric[1735]
p : numeric[1734]*

On the other hand, it's normal since I have the slope between two points on
the ROC curve and so I have x points and x-1 slope values. How to get the
graph?!

All the best,
Camille

-
--
Camille Leclerc, Master student
Lab ESE, UMR CNRS 8079
Univ Paris-Sud
Bat 362
F-91405  Orsay Cedex FRANCE
--
View this message in context: 
http://r.789695.n4.nabble.com/ROC-Analysis-tp4469203p4503354.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC Analysis

[Camille Leclerc]

Hi everybody,

Pascal, your script works again but I want to calculate the LR otherwise.
I know the likelihood ratio is linked at the roc curve and so there are
different ways to calculate the LR.
The slope of an ROC curve can be defined in three ways:
(1) as the tangent at a particular point on the ROC curve corresponding to a
test value x
(2) as the slope between the origin 0 and the point on the ROC curve
corresponding to a test value x 
(3) as the slope between two points on the ROC curve corresponding to the
test values x and y

http://r.789695.n4.nabble.com/file/n4478233/LR.png 

But in my case, I want calculated the LR with the third way. 
So, LR (x,y) =(sensitivity (x)-sensitivity (y))/(specificity (y)-specificity
(x))
= (TPR(x)-TPR (y))/(FPR (x)-FPR (y))

It is possible ?!

All the best,
Camille


-
--
Camille Leclerc, Master student
Lab ESE, UMR CNRS 8079
Univ Paris-Sud
Bat 362
F-91405  Orsay Cedex FRANCE
--
View this message in context: 
http://r.789695.n4.nabble.com/ROC-Analysis-tp4469203p4478233.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC Analysis

[Camille Leclerc]

Hi,

I made a mistake, effectively Pascal your method given to the first message
is correct. I didn't use the good data, so it's sure that it could not work
! 

But, I have an another question : Can I obtain a same graph except that the
y-axis represents the likelihood ratio ?

All the best,
Camille

-
--
Camille Leclerc, Master student
Lab ESE, UMR CNRS 8079
Univ Paris-Sud
Bat 362
F-91405  Orsay Cedex FRANCE
--
View this message in context: 
http://r.789695.n4.nabble.com/ROC-Analysis-tp4469203p4475038.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC Analysis

[Camille Leclerc]

Hi,
Pascal, I tried to use your method which functions but the values of the
abscissa are not good. When I talked about “value”, I wanted mean the values
of my starting dataset that I have classified into two classes positive and
negative classes, such as this :

Value  Status  TPF   TNF
10001   
487 0   
852 1   
927 1   
799 1   
369 0   
…   …   

And for each value of my dataset, I want the tpf and tnf. 
With your method, I have an abscissa which has values between 0 and 1.

All the best,
Camille



-
--
Camille Leclerc, Master student
Lab ESE, UMR CNRS 8079
Univ Paris-Sud
Bat 362
F-91405  Orsay Cedex FRANCE
--
View this message in context: 
http://r.789695.n4.nabble.com/ROC-Analysis-tp4469203p4471300.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] ROC Analysis

[Camille Leclerc]

Hi everybody,

I have a data set with a value and a status (positive or negative case) and
I want make a ROC Analysis. So, with ROCR Package, I have got the ROC curve
(True Positive Fraction [tpf] according 1-True Negative Fraction [1-tnf]).

http://r.789695.n4.nabble.com/file/n4469203/01.png 

But, now I want a new graphic which show the sum of true positive fraction
and true negative fraction according each value on my data set (tpf + tnf
according the values). 

http://r.789695.n4.nabble.com/file/n4469203/02.png 

If you have an idea !

Thank you very much for all help,
Camille Leclerc

--
Camille Leclerc, Master student
Lab ESE, UMR CNRS 8079
Univ Paris-Sud
Bat 362
F-91405  Orsay Cedex FRANCE



--
View this message in context: 
http://r.789695.n4.nabble.com/ROC-Analysis-tp4469203p4469203.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC curve

[Jim Lemon]

On 01/30/2012 08:52 PM, Josiane NJIWA wrote:



Hello all,

I am very new to R and i am facing two problems. First i didn't succeed 
changing the konsole language in english even after trying the line command set 
language='en'.
I would like to plot ROC curves. I have a serie of 10 threshold tests that i do 
for 10 patients. The prediction for the patients is always the same but the 
status can change given to the considered threshold.
I have 11 columns of 10 rows, the first colums containing the10 lines of the 
predicted status of the patients (0=cured, 1=non cured). Then follow 10 columns 
(10 thresholds) containing the found status using the threshold.
Please do someone know how i can use those values with R to plot ROC curves?


Hi Josiane,
The function roc.from.table in the epicalc package will produce a 
basic ROC curve. You will have to get the table that you send to the 
function in the correct form. I think you will have to lay out the table 
like this:


Predicted   Above threshold
Threshold1  x   x
Threshold2  x   x
Threshold3  x   x
Threshold4  x   x
Threshold5  x   x
Threshold6  x   x
Threshold7  x   x
Threshold8  x   x
Threshold9  x   x
Threshold10 x   x

I may be quite wrong as I don't know exactly what the successive 
threshold values may mean. You will probably want to run the function 
with grid=FALSE.


Jim

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] ROC curve

[Josiane NJIWA]

Hello all,

I am very new to R and i am facing two problems. First i didn't succeed 
changing the konsole language in english even after trying the line command set 
language='en'.
I would like to plot ROC curves. I have a serie of 10 threshold tests that i do 
for 10 patients. The prediction for the patients is always the same but the 
status can change given to the considered threshold.
I have 11 columns of 10 rows, the first colums containing the10 lines of the 
predicted status of the patients (0=cured, 1=non cured). Then follow 10 columns 
(10 thresholds) containing the found status using the threshold.
Please do someone know how i can use those values with R to plot ROC curves?

I thank you for your understanding,

Josiane.


Everything should be made as simple as possible, but not simpler.Albert 
Einstein.
[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC curve

[Corey Dow-Hygelund]

Hi Josiane,

Concerning ROC curves, the package ROCR should do what you want to do.  Use
install.packages to add it to you library.

Getting you data into a text file format, use read.delim to read into an
data frame.  Once you have a data frame, you can use the methods in ROCR to
analyze the data.

Best,

Corey

On Mon, Jan 30, 2012 at 1:52 AM, Josiane NJIWA joa...@yahoo.com wrote:



 Hello all,

 I am very new to R and i am facing two problems. First i didn't succeed
 changing the konsole language in english even after trying the line command
 set language='en'.
 I would like to plot ROC curves. I have a serie of 10 threshold tests that
 i do for 10 patients. The prediction for the patients is always the same
 but the status can change given to the considered threshold.
 I have 11 columns of 10 rows, the first colums containing the10 lines of
 the predicted status of the patients (0=cured, 1=non cured). Then follow 10
 columns (10 thresholds) containing the found status using the threshold.
 Please do someone know how i can use those values with R to plot ROC
 curves?

 I thank you for your understanding,

 Josiane.


 Everything should be made as simple as possible, but not simpler.
  Albert Einstein.
[[alternative HTML version deleted]]

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.




-- 
*The mark of a successful man is one that has spent an entire day on the
bank of a river without feeling guilty about it.*

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC curve

[David Winsemius]

On Jan 30, 2012, at 4:52 AM, Josiane NJIWA wrote:




Hello all,

I am very new to R and i am facing two problems. First i didn't  
succeed changing the konsole language in english even after trying  
the line command set language='en'.


R is a functional language, so it shouldn't surprise you that issuing  
a command does not do what you apparently expected based on your  
experience with macro languages. You should read:


?locales


I would like to plot ROC curves. I have a serie of 10 threshold  
tests that i do for 10 patients. The prediction for the patients is  
always the same but the status can change given to the considered  
threshold.
I have 11 columns of 10 rows, the first colums containing the10  
lines of the predicted status of the patients (0=cured, 1=non  
cured). Then follow 10 columns (10 thresholds) containing the found  
status using the threshold.
Please do someone know how i can use those values with R to plot ROC  
curves?


I thank you for your understanding,

Josiane.


Everything should be made as simple as possible, but not  
simpler.Albert Einstein.

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


David Winsemius, MD
Heritage Laboratories
West Hartford, CT

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC plot for KNN

[AHmed]

Qian Liu littleduck24 at gmail.com writes:

 
 Hi I need some help with ploting the ROC for K-nearest neighbors. Since KNN
 is a non-parametric classification methods, the predicted value will be
 either 0 or 1.
 It will not be able to test for different cutoff to plot ROC. What is the
 package or functions I should use to plot ROC for KNN?
 
 Thanks.
 Qian
 
   [[alternative HTML version deleted]]
 
 


I m wondering if you found any help or any codes to plot roc curve 

Thanks.
Salmi

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] ROC plot for KNN

[Qian Liu]

Hi I need some help with ploting the ROC for K-nearest neighbors. Since KNN
is a non-parametric classification methods, the predicted value will be
either 0 or 1.
It will not be able to test for different cutoff to plot ROC. What is the
package or functions I should use to plot ROC for KNN?

Thanks.
Qian

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] ROC plot with SEC and SPC as input

[CZ]

Hi, 

I am working on a ROC plot problem.  Thanks in advance for any help. 

We run 20 cross validation (CV) on our data for each customized threshold,
i.e. from 0.1 to 0.9.
Then for these 9 threshold values, we get the overall performance, that is
we are able to get overall sensitivity and specificity across each 20CVs
using different threshold.  So we have in total 9 points to be plotted on a
ROC graph.

But the question is it seems that the ROC functions only read the prediction
label and the actual label for each case, instead of the SEC and the SPE
values.  So we are looking for a way to feed the SEC and SPC values as input
in order to plot ROC.  

Thanks.


--
View this message in context: 
http://r.789695.n4.nabble.com/ROC-plot-with-SEC-and-SPC-as-input-tp3652562p3652562.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] ROC curves

[Shun-Fu Chen]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC from R-SVM?

[Max Kuhn]

The objects functions for kernel methods are unrelated to the area
under the ROC curve. However, you can try to choose the cost and
kernel parameters to maximize the ROC AUC.

See the caret package, specifically the train function.

Max

On Mon, Feb 21, 2011 at 5:34 PM, Angel Russo angerusso1...@gmail.com wrote:
 *Hi,

 *Does anyone know how can I show an *ROC curve for R-SVM*? I understand in
 R-SVM we are not optimizing over SVM cost parameter. Any example ROC for
 R-SVM code or guidance can be really useful.

 Thanks, Angel.

        [[alternative HTML version deleted]]

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.




-- 

Max

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC from R-SVM?

[Angel Russo]

Hi Max and Andrew,

Thanks so much for your reply. Indeed I found your link last night using
steps shown below.

My first question is if the following two steps are right and AUC is 51% as
shown below.

My seond question is that currently I am using cost parameter=1 (the default
in R-SVM; http://www.stanford.edu/group/wonglab/RSVMpage/R-SVM.html). To
improve the AUC from ROC curve, can I can optimize the SVM cost function
(instead of keeping it fixed a 1) to get the mimimum LOO error in training
cross.validation and then draw the ROC from decision.values as Step2 below
using a cost parameter that gave mimimum cross.validation error in the
training data. Is that right?

Many thanks.
--

*Step 1: For obtaining ROC curve of test data I turned on prob=T option:*

 svmres.prob - svm(traindx[,resrsvm$SelInd], as.factor(traindy),
decision.values = TRUE)
 svmpred.prob - predict(svmres.prob, testdx[,resrsvm$SelInd],
decision.values = TRUE)
 print(confusionMatrix(svmpred.prob,testdy))
Confusion Matrix and Statistics

   Reference
Prediction  Resistant Sensitive
  Resistant 513
  Sensitive3788

 Accuracy : 0.6503


*Step 2: Actual ROC plot command using output from above and plot attached
as well as pdf (I am assuming the following says the AUC is 51.4):*
 library(ROCR)
 svm.roc - prediction(attributes(svmpred.prob)$decision.values, testdy)
 svm.auc - performance(svm.roc, 'tpr', 'fpr')
 aucsvm - performance(svm.roc, 'auc')
 pdf(file=roc_curve_rsvm_decval.pdf)
 plot(svm.auc)
 print(str(aucsvm))
 print(str(aucsvm))
Formal class 'performance' [package ROCR] with 6 slots
  ..@ x.name  : chr None
  ..@ y.name  : chr Area under the ROC curve
  ..@ alpha.name  : chr none
  ..@ x.values: list()
  ..@ y.values:List of 1
  .. ..$ : num 0.514
  ..@ alpha.values: list()

---



On Tue, Feb 22, 2011 at 4:23 PM, Andrew Ziem az...@us.ci.org wrote:

 In addition's to Max's suggestion about caret, look at ROCR which
 visualizes ROC charts for any binary classifier.  I have an example of
 e1071::SVN and ROCR here


 https://heuristically.wordpress.com/2009/12/23/compare-performance-machine-learning-classifiers-r/



 -Original Message-
 From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org]
 On Behalf Of Angel Russo
 Sent: Monday, February 21, 2011 3:34 PM
 To: r-help@r-project.org
 Subject: [R] ROC from R-SVM?

 *Hi,

 *Does anyone know how can I show an *ROC curve for R-SVM*? I understand in
 R-SVM we are not optimizing over SVM cost parameter. Any example ROC for
 R-SVM code or guidance can be really useful.

 Thanks, Angel.




[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC from R-SVM?

[Andrew Ziem]

In addition's to Max's suggestion about caret, look at ROCR which visualizes 
ROC charts for any binary classifier.  I have an example of e1071::SVN and ROCR 
here

https://heuristically.wordpress.com/2009/12/23/compare-performance-machine-learning-classifiers-r/



-Original Message-
From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On 
Behalf Of Angel Russo
Sent: Monday, February 21, 2011 3:34 PM
To: r-help@r-project.org
Subject: [R] ROC from R-SVM?

*Hi,

*Does anyone know how can I show an *ROC curve for R-SVM*? I understand in
R-SVM we are not optimizing over SVM cost parameter. Any example ROC for
R-SVM code or guidance can be really useful.

Thanks, Angel.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] ROC from R-SVM?

[Angel Russo]

*Hi,

*Does anyone know how can I show an *ROC curve for R-SVM*? I understand in
R-SVM we are not optimizing over SVM cost parameter. Any example ROC for
R-SVM code or guidance can be really useful.

Thanks, Angel.

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] ROC curve in R

[ashu6886]

Hi,

i have a fairly large amount of genomic data. I have created a dataframe
which has Reference as one column and Variation as another. I want to
plot a ROC curve based on these 2 columns. I have serached the R manual but
I could not understand. Can anybody help me with the R code for plotting ROC
curve. 

Thnx
ashu6886
-- 
View this message in context: 
http://r.789695.n4.nabble.com/ROC-curve-in-R-tp2275431p2275431.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC curve in R

[David Winsemius]

On Jul 1, 2010, at 12:50 PM, ashu6886 wrote:



Hi,

i have a fairly large amount of genomic data. I have created a  
dataframe
which has Reference as one column and Variation as another. I  
want to
plot a ROC curve based on these 2 columns. I have serached the R  
manual but
I could not understand. Can anybody help me with the R code for  
plotting ROC

curve.



Not sure what sort of R manual you consulted, but surely any basic  
introduction must have covered how to search. In the interest of  
moving you a bit faster perhaps:


?RSiteSearch

--

David Winsemius, MD
West Hartford, CT

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC curve in R

[Bert Gunter]

Did you try googling on R ROC function (or something similar) or using the
RSiteSearch() function?

?RSiteSearch

RSiteSearch(ROC,restr=func)

Learn to use R's various search capabilities before posting, please!

Bert Gunter
Genentech Nonclinical Biostatistics
 
 

-Original Message-
From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On
Behalf Of ashu6886
Sent: Thursday, July 01, 2010 9:51 AM
To: r-help@r-project.org
Subject: [R] ROC curve in R


Hi,

i have a fairly large amount of genomic data. I have created a dataframe
which has Reference as one column and Variation as another. I want to
plot a ROC curve based on these 2 columns. I have serached the R manual but
I could not understand. Can anybody help me with the R code for plotting ROC
curve. 

Thnx
ashu6886
-- 
View this message in context:
http://r.789695.n4.nabble.com/ROC-curve-in-R-tp2275431p2275431.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC curve in R

[Saeed Abu Nimeh]

Try the ROCR package. http://rocr.bioinf.mpi-sb.mpg.de/ROCR.pdf
Saeed

On Thu, Jul 1, 2010 at 9:50 AM, ashu6886 ashu.infy.m...@gmail.com wrote:

 Hi,

 i have a fairly large amount of genomic data. I have created a dataframe
 which has Reference as one column and Variation as another. I want to
 plot a ROC curve based on these 2 columns. I have serached the R manual but
 I could not understand. Can anybody help me with the R code for plotting ROC
 curve.

 Thnx
 ashu6886
 --
 View this message in context: 
 http://r.789695.n4.nabble.com/ROC-curve-in-R-tp2275431p2275431.html
 Sent from the R help mailing list archive at Nabble.com.

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.


__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC curve in R

[Tal Galili]

Regarding search,

There is also the sos packege.

And yesterday someone wrote a nice R code to enable searching inside
the description of all packages:
http://www.r-bloggers.com/cran-search/


Cheers,
Tal




Contact
Details:---
Contact me: tal.gal...@gmail.com |  972-52-7275845
Read me: www.talgalili.com (Hebrew) | www.biostatistics.co.il (Hebrew) |
www.r-statistics.com (English)
--




On Thu, Jul 1, 2010 at 8:09 PM, Bert Gunter gunter.ber...@gene.com wrote:

 Did you try googling on R ROC function (or something similar) or using
 the
 RSiteSearch() function?

 ?RSiteSearch

 RSiteSearch(ROC,restr=func)

 Learn to use R's various search capabilities before posting, please!

 Bert Gunter
 Genentech Nonclinical Biostatistics



 -Original Message-
 From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org]
 On
 Behalf Of ashu6886
 Sent: Thursday, July 01, 2010 9:51 AM
 To: r-help@r-project.org
 Subject: [R] ROC curve in R


 Hi,

 i have a fairly large amount of genomic data. I have created a dataframe
 which has Reference as one column and Variation as another. I want to
 plot a ROC curve based on these 2 columns. I have serached the R manual but
 I could not understand. Can anybody help me with the R code for plotting
 ROC
 curve.

 Thnx
 ashu6886
 --
 View this message in context:
 http://r.789695.n4.nabble.com/ROC-curve-in-R-tp2275431p2275431.html
 Sent from the R help mailing list archive at Nabble.com.

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.


[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC curve in R

[Changbin Du]

Read the ROCR package, it is very good.





On Thu, Jul 1, 2010 at 9:50 AM, ashu6886 ashu.infy.m...@gmail.com wrote:


 Hi,

 i have a fairly large amount of genomic data. I have created a dataframe
 which has Reference as one column and Variation as another. I want to
 plot a ROC curve based on these 2 columns. I have serached the R manual but
 I could not understand. Can anybody help me with the R code for plotting
 ROC
 curve.

 Thnx
 ashu6886
 --
 View this message in context:
 http://r.789695.n4.nabble.com/ROC-curve-in-R-tp2275431p2275431.html
 Sent from the R help mailing list archive at Nabble.com.

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.




-- 
Sincerely,
Changbin
--

Changbin Du
DOE Joint Genome Institute
Bldg 400 Rm 457
2800 Mitchell Dr
Walnut Creet, CA 94598
Phone: 925-927-2856

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC curve in R

[Frank E Harrell Jr]

On 07/01/2010 01:33 PM, Changbin Du wrote:

Read the ROCR package, it is very good.


Just be sure you really need an ROC curve.  More often than not it gets 
in the way of understanding.


Frank







On Thu, Jul 1, 2010 at 9:50 AM, ashu6886ashu.infy.m...@gmail.com  wrote:



Hi,

i have a fairly large amount of genomic data. I have created a dataframe
which has Reference as one column and Variation as another. I want to
plot a ROC curve based on these 2 columns. I have serached the R manual but
I could not understand. Can anybody help me with the R code for plotting
ROC
curve.

Thnx
ashu6886
--
View this message in context:
http://r.789695.n4.nabble.com/ROC-curve-in-R-tp2275431p2275431.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.








--
Frank E Harrell Jr   Professor and ChairmanSchool of Medicine
 Department of Biostatistics   Vanderbilt University

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC curve

[Claudia Beleites]

Dear Changbin,


I want to know how to select the optimal decision threshold from the ROC
curve? 

Depends on what optimal means. I think there are a bunch of different criteria 
used:

- point closest to the ideal model
- point furthest from the guessing model
- these criteria may include costs, i.e. a FP/FN ratio != 1
- ...

More practical:
If you use ROCR: the help of the performance class explains the slots in the 
object. You find there the data of the curve, incl. the thresholds.



At what threshold will give the highest accuracy?

to know that, optmize the accuracy as function of the threshold.

Remember: finding the optimal threshold from a ROC curve is a data-driven 
optimization. You need to validate the resulting model with independent test 
data afterwards.




--
Claudia Beleites
Dipartimento dei Materiali e delle Risorse Naturali
Università degli Studi di Trieste
Via Alfonso Valerio 6/a
I-34127 Trieste

phone: +39 0 40 5 58-37 68
email: cbelei...@units.it

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC curve

[Frank E Harrell Jr]

On 05/24/2010 02:14 AM, Claudia Beleites wrote:

Dear Changbin,


I want to know how to select the optimal decision threshold from the ROC
curve?

Depends on what optimal means. I think there are a bunch of different
criteria used:

- point closest to the ideal model
- point furthest from the guessing model
- these criteria may include costs, i.e. a FP/FN ratio != 1
- ...

More practical:
If you use ROCR: the help of the performance class explains the slots in
the object. You find there the data of the curve, incl. the thresholds.


At what threshold will give the highest accuracy?

to know that, optmize the accuracy as function of the threshold.

Remember: finding the optimal threshold from a ROC curve is a
data-driven optimization. You need to validate the resulting model with
independent test data afterwards.


That point is excellent.  In addition, such decision analysis assumes 
that (1) a forced yes/no decision is acceptable, i.e., a predicted 
probability in the middle is forced to be categorized as low or high 
as opposed to no decision; get more data, and (2) the 
utility/cost/loss function is identical across subjects (which it almost 
never is).


Frank

--
Frank E Harrell Jr   Professor and ChairmanSchool of Medicine
 Department of Biostatistics   Vanderbilt University

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] ROC curve

[Changbin Du]

HI, Dear R community,

I want to know how to select the optimal decision threshold from the ROC
curve? At what threshold will give the highest accuracy?

Thanks!

-- 
Sincerely,
Changbin
--

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] ROC curve in randomForest

[Changbin Du]

require(randomForest)

rf.pred-predict(fit, valid, type=prob)
 rf.pred[1:20, ]
 0  1
16  0. 1.
23  0.3158 0.6842
43  0.3030 0.6970
52  0.0886 0.9114
55  0.1216 0.8784
75  0.0920 0.9080
82  0.4332 0.5668
120 0.2302 0.7698
128 0.1336 0.8664
147 0.4272 0.5728
148 0.0490 0.9510
153 0.0556 0.9444
161 0.0760 0.9240
162 0.4564 0.5436
172 0.5148 0.4852
176 0.1730 0.8270
215 0.0100 0.9900
222 0.0104 0.9896
255 0.1358 0.8642
267 0.1000 0.9000
 rf.pred-predict(fit, valid, type=prob)

 rf.roc - prediction(rf.pred[,2], valid[,32])
 rf.auc - performance(rf.roc, 'tpr', 'fpr')
 plot(rf.auc)

Dear R community,

I am plot the ROC curve for randomforest result, I am not sure the above
code is right or not?

I just used the second column of the probabilities, the pr(outcome=1).

Can anyone give some comments?


someguys use the following codes to plot, but I can not use the treeresponse
commad in randomforest object directly.


# create model using random forest and bagging ensemble using conditional
inference trees
035x.cf - cforest(Class ~ ., data=BreastCancer[ind == 1,], control =
cforest_unbiased(mtry = ncol(BreastCancer)-2))
036x.cf.pred - predict(x.cf, newdata=BreastCancer[ind == 2,])
037x.cf.prob -  1- unlist(treeresponse(x.cf, BreastCancer[ind == 2,]),
use.names=F)[seq(1,nrow(BreastCancer[ind == 2,])*2,2)]
038









-- 
Sincerely,
Changbin
--

Changbin Du
DOE Joint Genome Institute
Bldg 400 Rm 457
2800 Mitchell Dr
Walnut Creet, CA 94598
Phone: 925-927-2856

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] ROC curve in randomSurvivalForest

[Nick Fankhauser]

I'm using randomSurvivalForest to predict survival from a rather small 
sample. As it's not enough data to have training and validation set, I 
rely on the Estimate of error rate computed by the randomForest. If I 
understand the method correctly, it repeatedly partitions the data into 
varying training/validation sets during the learning steps, which also 
produces the estimate of error.
My questions is, would it be possible to compute a ROC curve during RF 
training?
A possible approach I considered would be to train the RF on a subset of 
the data and create a ROC curve from the prediction on the remaining 
data. By repeating this process, I would get the variation of the ROC 
curve for the different possible data subsets.
But this seems to be not such an elegant solution, as this could be done 
during one instance of RF learning.
It would be very helpful, if someone could point to an approach on how 
to do this.


Nick Fankhauser

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] roc plot with zero length labels error

[Uwe Ligges]

[Going through some old and from what I see unanswered messages

Hard to tell what is going on without telling us the data. Works with 
the examples, so may be related to one of the objects scicomp5.model1, 
pred, T, or auc.



 PLEASE do read the posting guide 
http://www.R-project.org/posting-guide.html

 and provide commented, minimal, self-contained, reproducible code.


Best wishes,
Uwe Ligges




Chris Anderson wrote:
I am trying to create the roc plot bootstrap method from library(verification), and when I set the plot =both or emp then I get the following error. The roc.plot works fine when the plot is set to binorm. This is my first time using this function in R and am not sure what this error means or how to resolve the issue. It seems to work ok with the example dataset. Is there an option or data formatting I need to due to resolve this error? 
 
roc.plot((scicomp5.model1[164]==High Price), pred, CI = T, n.boot = 100, plot = 'both',binormal = TRUE,

+  main = paste('AREA UNDER CURVE (AUC) =', round(auc$A, 2)))
Error in text.default(DAT[id, 3, i], DAT[id, 2, i], plot.thres[id2], pos = 4,  : 
  zero length 'labels'




Chris Anderson

http://www.seocodebreaker.com/?thankyou-page=429

Instant Debt Consolidation Quote
Free debt consolidation quote online! No obligation. No credit check.
http://thirdpartyoffers.netzero.net/TGL2241/c?cp=cyoog8NQa4mNZIgs6R2NrwAAJ1HbVVWM9bC-t-Va2hgXndJzAAQFAEWLnT8AAAMlABI2kQA=
[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] ROC

[carol white]

My question would be related more to time-dependent ROC (survivalROC package):

I would like to know how true positives (TP) and false positives (FP) are 
related to the order of survival times and diagnostic markers. That is, having 
the a set of TP and FP, each of them is calculated from which diagnostic marker 
and survival times. Let's take mayo data set. How could I match the TP and FP 
generated from survivalROC function to cut.values (output) and Stimes (input)?

Best,

Carol

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] roc plot with zero length labels error

[Chris Anderson]

I am trying to create the roc plot bootstrap method from library(verification), 
and when I set the plot =both or emp then I get the following error. The 
roc.plot works fine when the plot is set to binorm. This is my first time using 
this function in R and am not sure what this error means or how to resolve the 
issue. It seems to work ok with the example dataset. Is there an option or data 
formatting I need to due to resolve this error? 
 
roc.plot((scicomp5.model1[164]==High Price), pred, CI = T, n.boot = 100, plot 
= 'both',binormal = TRUE,
+  main = paste('AREA UNDER CURVE (AUC) =', round(auc$A, 2)))
Error in text.default(DAT[id, 3, i], DAT[id, 2, i], plot.thres[id2], pos = 4,  
: 
  zero length 'labels'



Chris Anderson

http://www.seocodebreaker.com/?thankyou-page=429

Instant Debt Consolidation Quote
Free debt consolidation quote online! No obligation. No credit check.
http://thirdpartyoffers.netzero.net/TGL2241/c?cp=cyoog8NQa4mNZIgs6R2NrwAAJ1HbVVWM9bC-t-Va2hgXndJzAAQFAEWLnT8AAAMlABI2kQA=
[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] ROC curve and gains/lift chart

[clue_less]

What is the difference between ROC curve and gains/lift chart?

how to do them in R?

Thanks.
-- 
View this message in context: 
http://www.nabble.com/ROC-curve-and-gains-lift-chart-tp25083979p25083979.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC curve and gains/lift chart

[Christian Schulz]

Have look in the ROCR package and Example's.

HTH Christian


What is the difference between ROC curve and gains/lift chart?

how to do them in R?

Thanks.



__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] ROC curve using epicalc (after logistic regression) (re-sent)

[Clifford Long]

Dear R-help,

I am resending as I believe I screwed up the e-mail address to R-help
earlier.  Sorry for my lack of attention to detail, and for any
inconvenience.

I have also sent the question to the package maintainer, as suggested
in the posting guide.

Regards,

Cliff



-- Forwarded message --
From: Clifford Long gnolff...@gmail.com
Date: Sun, Jul 26, 2009 at 8:46 PM
Subject: Fwd: ROC curve using epicalc (after logistic regression)
To: cvira...@medicine.psu.ac.th


Dear Virasakdi Chongsuvivatwong,

After sending the message below to the R-help mailing list, it
occurred to me that I probably should also have sent a copy to you,
per R posting guidance.

I would be interested in any thoughts or suggestions that you might
have regarding my difficulty using the ROCR routine in the epicalc
package.  (I've used this before, and find it to be a very helpful
package ... thanks.)

Is my issue related to the way the data is structured for the glm
routine - meaning not with individual cases, but instead by counts
(per DOE treatment) of pass, fail, and total?

Or perhaps I've made another error?

I'll understand if you don't have the time to look this over.  In case
you do, any direction/guidance will be appreciated.

Thank you for your time, and for this excellent package.

Regards,

Cliff Long




-- Forwarded message --
From: Clifford Long gnolff...@gmail.com
Date: Sun, Jul 26, 2009 at 3:52 PM
Subject: ROC curve using epicalc (after logistic regression)
To: R-help@r-project.org


Dear R-help list,

I'm attempting to use the ROC routine from the epicalc package after
performing a logistic regression analysis.  My code is included after
the sessionInfo() result.  The datafile (GasketMelt1.csv) is attached.
 I updated both R and the epicalc packages and tried again before
sending this request.

sessionInfo result:

R version 2.9.1 (2009-06-26)
i386-pc-mingw32

locale:
LC_COLLATE=English_United States.1252;LC_CTYPE=English_United
States.1252;LC_MONETARY=English_United
States.1252;LC_NUMERIC=C;LC_TIME=English_United States.1252

attached base packages:
[1] splines   stats     graphics  grDevices utils     datasets  methods
[8] base

other attached packages:
[1] caret_4.19      lattice_0.17-25 epicalc_2.9.1.2 survival_2.35-4
[5] foreign_0.8-36

loaded via a namespace (and not attached):
[1] grid_2.9.1  tools_2.9.1


Header information from package 'epicalc':
Package:            epicalc
Version:            2.9.1.2
Date:               2009-07-14


My code ...

#
#  Logistic Regression   (the model result is as expected)
#

dfile = 'GasketMelt1.csv'
gmelt.df = read.csv(dfile, header = TRUE, as.is = TRUE)
names(gmelt.df)

gmelt.df$p = gmelt.df$Pass / gmelt.df$Total

gmelt.glm = glm(p ~ Time + Temperature + Depth
                       + Time*Temperature + Time*Depth + Temperature*Depth,
                       family = binomial(link = logit), data=gmelt.df,
weight=Total)
summary(gmelt.glm)

#
#  ROC
#
library(epicalc)

lroc(gmelt.glm, graph = TRUE, line.col = red)


The error message:

 lroc(gmelt.glm, graph = TRUE, line.col = red)
Error in dimnames(x) - dn :
 length of 'dimnames' [2] not equal to array extent



Have I overlooked something?


Many thanks to anyone who might have a suggestion.

Cliff
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] ROC curve using epicalc (after logistic regression)

[Clifford Long]

Dear R-help list,

I'm attempting to use the ROC routine from the epicalc package after
performing a logistic regression analysis.  My code is included after
the sessionInfo() result.  The datafile (GasketMelt1.csv) is attached.
 I updated both R and the epicalc packages and tried again before
sending this request.

sessionInfo result:

R version 2.9.1 (2009-06-26)
i386-pc-mingw32

locale:
LC_COLLATE=English_United States.1252;LC_CTYPE=English_United
States.1252;LC_MONETARY=English_United
States.1252;LC_NUMERIC=C;LC_TIME=English_United States.1252

attached base packages:
[1] splines   stats graphics  grDevices utils datasets  methods
[8] base

other attached packages:
[1] caret_4.19  lattice_0.17-25 epicalc_2.9.1.2 survival_2.35-4
[5] foreign_0.8-36

loaded via a namespace (and not attached):
[1] grid_2.9.1  tools_2.9.1


Header information from package 'epicalc':
Package:epicalc
Version:2.9.1.2
Date:   2009-07-14


My code ...

#
#  Logistic Regression   (the model result is as expected)
#

dfile = 'GasketMelt1.csv'
gmelt.df = read.csv(dfile, header = TRUE, as.is = TRUE)
names(gmelt.df)

gmelt.df$p = gmelt.df$Pass / gmelt.df$Total

gmelt.glm = glm(p ~ Time + Temperature + Depth
+ Time*Temperature + Time*Depth + Temperature*Depth,
family = binomial(link = logit), data=gmelt.df, 
weight=Total)
summary(gmelt.glm)

#
#  ROC
#
library(epicalc)

lroc(gmelt.glm, graph = TRUE, line.col = red)


The error message:

 lroc(gmelt.glm, graph = TRUE, line.col = red)
Error in dimnames(x) - dn :
  length of 'dimnames' [2] not equal to array extent



Have I overlooked something?


Many thanks to anyone who might have a suggestion.

Cliff
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Roc curves confidence intervals

[marc bernard]

Dear all, I am looking for an R package that allows me to calculate and plot 
the confidence intervals for the roc curve using for example some 
bootstrapping. I tried ROCR who seems doing such work but i couldn't find the 
right option in it. Many thanks Bests Marc
_


[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Roc curves confidence intervals

[David Freedman]

Would you be interested in the cross-validation that's on pp 3 and 4 of the
ROCR package PDF?

plot(perf,lwd=3,avg=vertical,spread.estimate=boxplot,add=TRUE)

there are various options for the 'spread.estimate'

David Freedman

marc bernard-2 wrote:
 
 
 Dear all, I am looking for an R package that allows me to calculate and
 plot the confidence intervals for the roc curve using for example some
 bootstrapping. I tried ROCR who seems doing such work but i couldn't
 find the right option in it. Many thanks Bests Marc
 _
 
 
   [[alternative HTML version deleted]]
 
 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.
 
 

-- 
View this message in context: 
http://www.nabble.com/Roc-curves-confidence-intervals-tp21850816p21851991.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] ROC curve from logistic regression

[gallon li]

I know how to compute the ROC curve and the empirical AUC from the logistic
regression after fitting the model.

But here is my question, how can I compute the standard error for the AUC
estimator resulting form logistic regression? The variance should be more
complicated than AUC based on known test results. Does anybody know a
reference on this problem?

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC curve from logistic regression

[Frank E Harrell Jr]

gallon li wrote:

I know how to compute the ROC curve and the empirical AUC from the logistic
regression after fitting the model.

But here is my question, how can I compute the standard error for the AUC
estimator resulting form logistic regression? The variance should be more
complicated than AUC based on known test results. Does anybody know a
reference on this problem?



The rcorr.cens function in the Hmisc package will compute the std. error 
of Somers' Dxy rank correlation.  Dxy = 2*(C-.5) where C is the ROC 
area.  This standard error does not include a variance component for the 
uncertainty in the model (e.g., it does not penalize for the estimation 
of the regression coefficients if you are estimating the coefficients 
and assessing ROC area on the same sample).


The lrm function in the Design package fits binary and ordinal logistic 
regression models and reports C, Dxy, and other measures.


I haven't seen an example where drawing the ROC curve provides useful 
information that leads to correct actions.


Frank
--
Frank E Harrell Jr   Professor and Chair   School of Medicine
 Department of Biostatistics   Vanderbilt University

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC curve from logistic regression

[Pedro.Rodriguez]

Hi

Try the following reference:

Comparison of Three Methods for Estimating the
Standard Error of the Area under the Curve in ROC
Analysis of Quantitative Data by Hajian-Tilaki and Hanley, Academic
Radiology, Vol 9, No 11, November 2002.

Below is a simple implementation that will return both the AUC and its
standard error (DeLong et al method). 

Hope this helps...

Pedro


#Input: yreal [-1,1]

auc - function(yreal,forecasts){

sizeT -nrow(yreal)
pos - 0
for(i in 1:sizeT){
if(yreal[i]0) {pos - pos + 1}
}
neg - sizeT-pos
yrealpos - vector(length=pos)
yrealneg - vector(length=neg)
forepos  - vector(length=pos)
foreneg  - vector(length=neg)

controlpos - 1
controlneg - 1
for(i in 1:sizeT){
if(yreal[i]0) {
yrealpos[controlpos] - yreal[i]
forepos[controlpos]  - forecasts[i]
controlpos - controlpos + 1
} else {
yrealneg[controlneg] - yreal[i]
foreneg[controlneg] - forecasts[i]
controlneg - controlneg + 1
}
}
oper - 0
for( i in 1:pos){
for(j in 1:neg){
if(forepos[i]  foreneg[j]) {oper - oper + 1}
if(forepos[i]==foreneg[j]) {oper - oper + 0.50
} 
}
}

area - oper/(pos*neg)
vpj - vector(length=pos)
vqk - vector(length=neg)
oper - 0
for(i in 1:pos){
for(j in 1:neg){
if(forepos[i]  foreneg[j]) {oper -
oper + 1 
} else {if(forepos[i]==foreneg[j]) {oper
- oper + 0.50 }} 
}
division - oper/neg
resta - (division-area)^2
vpj[i] - resta
oper - 0
}
oper  - 0
resta - 0
for(j in 1:neg){
for(i in 1:pos){
if(forepos[i]  foreneg[j]) {oper -
oper + 1 
} else {if(forepos[i]==foreneg[j]) {oper
- oper + 0.50 }} 
}
division - oper/pos
resta - (division-area)^2  
vqk[j] - resta
oper   - 0
}
vpj - vpj/(pos*(pos-1))
vqk - vqk/(neg*(neg-1))
var - sum(vpj)+sum(vqk)
s   - sqrt(var)

return(list(AUC=area, std=s))
}

-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
On Behalf Of Frank E Harrell Jr
Sent: Monday, September 08, 2008 8:22 AM
To: gallon li
Cc: r-help
Subject: Re: [R] ROC curve from logistic regression

gallon li wrote:
 I know how to compute the ROC curve and the empirical AUC from the
logistic
 regression after fitting the model.
 
 But here is my question, how can I compute the standard error for the
AUC
 estimator resulting form logistic regression? The variance should be
more
 complicated than AUC based on known test results. Does anybody know a
 reference on this problem?


The rcorr.cens function in the Hmisc package will compute the std. error

of Somers' Dxy rank correlation.  Dxy = 2*(C-.5) where C is the ROC 
area.  This standard error does not include a variance component for the

uncertainty in the model (e.g., it does not penalize for the estimation 
of the regression coefficients if you are estimating the coefficients 
and assessing ROC area on the same sample).

The lrm function in the Design package fits binary and ordinal logistic 
regression models and reports C, Dxy, and other measures.

I haven't seen an example where drawing the ROC curve provides useful 
information that leads to correct actions.

Frank
-- 
Frank E Harrell Jr   Professor and Chair   School of Medicine
  Department of Biostatistics   Vanderbilt
University

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC curve from logistic regression

[Jorge Ivan Velez]

Hi there,
See ?lroc in the epicalc package.

HTH,


Jorge



On Mon, Sep 8, 2008 at 4:02 AM, gallon li [EMAIL PROTECTED] wrote:

 I know how to compute the ROC curve and the empirical AUC from the logistic
 regression after fitting the model.

 But here is my question, how can I compute the standard error for the AUC
 estimator resulting form logistic regression? The variance should be more
 complicated than AUC based on known test results. Does anybody know a
 reference on this problem?

[[alternative HTML version deleted]]

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.


[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] ROC analysis

[Eleni Christodoulou]

Hello list,

I am trying to perform ROC analysis and count the AUC in order to validate
my results. I use package ROCR. I would like to count the AUC not under the
cutoff found by performance but to use another cutoff that I calculate.
How could I change the following  command in order to get what I want?
perform=performance(pred,measure=auc,x.measure=cutoff), where pred is a
prediction object.

Thank you very much,
Eleni

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC analysis

[Richard Pearson]

Eleni

Does the fpr.stop argument do what you want? This is described in 
?performance under the details of the auc measure. Try, e.g.

perform=performance(pred,measure=auc,fpr.stop=0.5)


Richard.


Eleni Christodoulou wrote:
 Hello list,

 I am trying to perform ROC analysis and count the AUC in order to validate
 my results. I use package ROCR. I would like to count the AUC not under the
 cutoff found by performance but to use another cutoff that I calculate.
 How could I change the following  command in order to get what I want?
 perform=performance(pred,measure=auc,x.measure=cutoff), where pred is a
 prediction object.

 Thank you very much,
 Eleni

   [[alternative HTML version deleted]]

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.



__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC analysis

[Eleni Christodoulou]

Richard, thanks,  I think it will work. I will calculate the cutoff value
and then, from the prediction object, find the fpr that is related to it and
put it as argument to performance. I will keep you informed.

Eleni

On Wed, Mar 19, 2008 at 11:51 AM, Richard Pearson 
[EMAIL PROTECTED] wrote:

 Eleni

 Does the fpr.stop argument do what you want? This is described in
 ?performance under the details of the auc measure. Try, e.g.

 perform=performance(pred,measure=auc,fpr.stop=0.5)


 Richard.


 Eleni Christodoulou wrote:
  Hello list,
 
  I am trying to perform ROC analysis and count the AUC in order to
 validate
  my results. I use package ROCR. I would like to count the AUC not under
 the
  cutoff found by performance but to use another cutoff that I
 calculate.
  How could I change the following  command in order to get what I want?
  perform=performance(pred,measure=auc,x.measure=cutoff), where pred
 is a
  prediction object.
 
  Thank you very much,
  Eleni
 
[[alternative HTML version deleted]]
 
  __
  R-help@r-project.org mailing list
  https://stat.ethz.ch/mailman/listinfo/r-help
  PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
  and provide commented, minimal, self-contained, reproducible code.
 
 

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.


[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] ROC analysis

[Frank E Harrell Jr]

Eleni Christodoulou wrote:
 Richard, thanks,  I think it will work. I will calculate the cutoff value
 and then, from the prediction object, find the fpr that is related to it and
 put it as argument to performance. I will keep you informed.
 
 Eleni

You will probably need to bootstrap the entire process to get a decent 
estimate of fpr as your cutoff was not specified in advance.

Also note that you are implicitly assuming a constant utility function 
across all subjects because you are not using probabilities but are 
ignoring information by creating only a binary prediction.

Frank Harrell

 
 On Wed, Mar 19, 2008 at 11:51 AM, Richard Pearson 
 [EMAIL PROTECTED] wrote:
 
 Eleni

 Does the fpr.stop argument do what you want? This is described in
 ?performance under the details of the auc measure. Try, e.g.

 perform=performance(pred,measure=auc,fpr.stop=0.5)


 Richard.


 Eleni Christodoulou wrote:
 Hello list,

 I am trying to perform ROC analysis and count the AUC in order to
 validate
 my results. I use package ROCR. I would like to count the AUC not under
 the
 cutoff found by performance but to use another cutoff that I
 calculate.
 How could I change the following  command in order to get what I want?
 perform=performance(pred,measure=auc,x.measure=cutoff), where pred
 is a
 prediction object.

 Thank you very much,
 Eleni

   [[alternative HTML version deleted]]

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.


 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.

 
   [[alternative HTML version deleted]]
 
 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.
 


-- 
Frank E Harrell Jr   Professor and Chair   School of Medicine
  Department of Biostatistics   Vanderbilt University

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.