David,
I just saw Rene's note about the -r 0.25 decoy ratio. I'm similarly using 2
decoy sets (50% target, 25% DECOY_1, 25% DECOY_2) but with -r 0.5. I had
assumed the ratio was supposed to be specified as decoys_used/targets and
there are twice as many targets as DECOY_2s in my case so -r = 0.5.
Having looked in ProphetModels.pl I'm now not so sure.... the estimation if
-r isn't supplied is pp_prob_array / pp_prob_array_decoy for hits with
p<=0.02, but I'm not sure whether this is total/decoy or target/decoy.
Can you confirm which approach is correct?
Not a huge problem for me if -r 0.5 is wrong, as am computing and using
decoy stats elsewhere, external to TPP. Would just mean the plots from
ProphetModels.pl that are being saved are wrong.
Thanks,
Dave Trudgian
On Thursday, December 19, 2013 2:01:53 AM UTC-6, Rene B wrote:
>
> Hi David,
>
> Thank you for your quick reply and suggestions. The decoy ratio is set to
> 0.25 as I use two sets of decoys, one for modeling and the other for
> validation. Each decoy set corresponds to 25% of entries in the database.
>
> Kind regards,
>
> Rene
>
>
> Op woensdag 18 december 2013 20:13:27 UTC+1 schreef David Shteynberg:
>>
>> Hello Rene
>>
>> Thanks for using the tools and double checking your work.
>>
>> In my tests I have found that applying the NSP model at the iProphet step
>> greatly improves performance on peptide level. And applying the NSP model
>> at the ProteinProphet step improves performance on the protein level. The
>> two models are somewhat different since the ProteinProphet model considers
>> grouping information while the iProphet model doesnt. I have not found the
>> two to interfere.
>>
>> A safe and conservative approach so would look at the conservative
>> estimate e.g. ProteinProphet probability cutoff to give me 1% error with
>> decoys or 1% error with the model which ever is more conservative.
>>
>> When the model tends to underestimate error on protein or peptide level
>> this is usually stemming from underestimation at the spectrum level by
>> PeptideProphet and can be controlled by the CLEVEL={value} option for
>> PeptideProphetParser -c{value} for xinteract. Setting this to a number
>> greater than zero like .5 or 1 or 2 will serve to make the model more
>> conservative overall, a negative value will have opposite effect which will
>> carry through to the peptide and protein levels.
>>
>> Also I am curious why you set decoy rate to 0.25?
>>
>> Best,
>> David
>> On Dec 18, 2013 7:29 AM, "Rene B" <[email protected]> wrote:
>>
>>> Hi all,
>>>
>>> I am running PeptideProphet, iProphet and ProteinProphet (TPP 4.6.3) on
>>> Q Exactive data searched with Comet, Myrimatch and OMSSA. I wondered if the
>>> NSP model should be disabled in ProteinProphet when it is enabled in
>>> iProphet? I got confused because it seems Petunia enables the NSP model
>>> both in iprophet and proteinprophet by default (ie. when xinteract runs
>>> with the -ip option).
>>>
>>> Another question is that when I compare decoy estimated protein FDRs to
>>> ProteinProphet modelled FDRs, ProteinProphet seems a bit optimistic (decoy
>>> based FDR of 0.1% corresponds to ~0.02% model FDR). This is with NSP
>>> enabled in iProphet and disabled in ProteinProphet. How should I deal with
>>> discrepancy, ie. should I take the decoy or probability based FDR to select
>>> a probability cutoff?
>>>
>>> I have attached some examples for a search with myrimatch only. These
>>> are the commands I used to generate the graphs:
>>>
>>> xinteract -Nmyrimatch.pep.xml -OAP -p0 -a%ExperimentFolder% -dDECOY0
>>> -E%ExperimentTag% *.pep.xml
>>> InterProphetParser myrimatch.pep.xml myrimatch.ipro.pep.xml
>>> ProphetModels.pl -i myrimatch.ipro.pep.xml -k -r 0.25 -d "DECOY1"
>>> ProteinProphet myrimatch.ipro.pep.xml myrimatch.prot.xml IPROPHET NONSP
>>> ProtProphModels.pl -k -r 0.25 -d DECOY1 -i myrimatch.prot.xml
>>>
>>> The graphs are:
>>>
>>> myrimatch_all.ipro.pep_FDR_10pc: PeptideProphet/iProphet decoy vs model
>>> FDR, all models enabled
>>> myrimatch_nonsp.ipro.pep_FDR_10pc: PeptideProphet/iProphet decoy vs
>>> model FDR, NSP model disabled in iProphet
>>> myrimatch_nonsp.prot_FDR_5pc: ProteinProphet decoy vs model FDR, NSP
>>> model disabled in ProteinProphet
>>> myrimatch_all.prot_FDR_5pc: ProteinProphet decoy vs model FDR, NSP model
>>> enabled in iProphet and ProteinProphet
>>>
>>> Thanks in advance!
>>>
>>> Kind regards,
>>>
>>> Rene
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "spctools-discuss" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> To post to this group, send email to [email protected].
>>> Visit this group at http://groups.google.com/group/spctools-discuss.
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>
>>
--
You received this message because you are subscribed to the Google Groups
"spctools-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/spctools-discuss.
For more options, visit https://groups.google.com/groups/opt_out.
