Hi David, The 1% FDR I used is decoy-based, which works out as using a protein prophet cutoff of ~0.8 (in both cases).
The dataset is large (~ 420,000 total spectra reported in the prot.xml) - perhaps this has an effect? Would it be useful for me to send across my prot.xml files (or pep.xmls?) Thanks Pete On Thursday, 21 June 2018 10:55:11 UTC-7, David Shteynberg wrote: > > Hi Pete, > > Thank you for the summary. I had a question about the 1% FDR. Is this > decoy-based or model-based? I am wondering what protein counts you will > observe when you compare at the 1% decoy-based FDR between running NSP in > both iProphet and ProteinProphet, running NSP only in iProphet and running > NSP only in ProteinProphet? > > Thank you, > -David > > > On Thu, Jun 21, 2018 at 10:39 AM, <pbell....@gmail.com <javascript:>> > wrote: > >> Hi David, >> >> Thanks a lot for your reply. >> >> Your first answer makes a big difference - I had been combining iprophet >> results for a second round which had previously combined multiple search >> engines. By combining the results from different search engines only in the >> second iProphet run, my numbers are now more consistent irrespective of >> order of combination. >> >> Regarding the second point, switching NSP off in protein prophet (after >> running NSP in iProphet) makes quite a big difference to my final protein >> numbers (2632 entries versus 2336 at 1% FDR). >> >> Taking a look at the additional matches, they are all single peptide >> sequence hits - however, from manual inspection of several of these though, >> there are often multiple matches with high peptide prophet scores across >> several different biological replicates, and the spectra look good. Some of >> them do look like single matches though (albeit with good looking spectra). >> >> Given that both are 1% FDR, it's difficult to choose the most appropriate >> method to choose. From what I've said, I think the risk of false negatives >> is greater (running both NSP models) than the risk of false positives >> (running iprophet NSP only) - my thoughts are that it will be better to use >> NSP 'off' in protein prophet, but to consider protein IDs from single >> peptide sequence hits as being less confident. >> >> Thanks again for your help, >> Pete >> >> >> >> >> On Wednesday, 20 June 2018 13:29:06 UTC-7, David Shteynberg wrote: >>> >>> Hello Pete, >>> >>> I think the answer to your first question is it depends on the specifics >>> of your analysis. >>> >>> You can pass iProphet files through iProphet again, since it will just >>> use the PeptideProphet probabilities which are not modified (only reported) >>> by iProphet. If the iProphet is from a single search engine this should be >>> just fine. However, if the iProphet file contains results from multiple >>> search engines then you probably don't want to combine it with iProphet >>> again as in this case each spectrum search result comes from the highest >>> scoring search engine, so not all the information will be available for >>> iProphet in the second analysis. Also, for your large analysis that is >>> currently failing in Petunia you might consider running the tool on the >>> commandline. >>> >>> For your second question, the NSP model can be disabled on the >>> commandline using the NONSP flag. The ProteinProphet NSP model is >>> implemented differently in iProphet and in ProteinProphet. Although, in >>> theory applying the models both times could be problematic due to their >>> similarity, in practice the models are different enought that during >>> testing I have not observed deleterious effects from using both the >>> iProphet and the ProteinProphet NSP models. You can try running the tools >>> in different ways and comparing the performance. Using both NSP models is >>> the current default and you would have to explicitly disable the models >>> when you run each tool. >>> >>> I hope this helps. >>> >>> -David >>> >>> >>> >>> >>> On Tue, Jun 19, 2018 at 5:29 PM, <pbell....@gmail.com> wrote: >>> >>>> Hi All, >>>> >>>> I have 2 questions I'd be grateful if people could help answer: >>>> >>>> 1) >>>> Is it valid to combine multiple iprophet.pep.xml files by passing >>>> through iprophet for a second time? - alternatively, is it valid to >>>> combine >>>> a single iprophet.pep.xml file with interact.pep.xml files in iprophet? >>>> >>>> I am trying to combine a lot of different experiments / search engines >>>> results etc, and have been combining in iprophet - but I appear to have >>>> maxed out the number interact.pep.xml files to pass into iprophet. Beyond >>>> a >>>> certain number of files (doesn't appear to be file-specific), iProphet >>>> fails. As a workaround, I wondered if I could simply run half of the files >>>> through iProphet, then combine the resulting file with the remaining files >>>> to be run, by running iprophet again prior to running protein prophet. - >>>> Would this be valid? >>>> >>>> 2) >>>> I attended a TPP course last year in which the course notes stated that >>>> NSP should be switched off in iProphet, if NSP model is to be used in >>>> protein prophet. >>>> >>>> I am using petunia (running protein prophet on the iprophet results), >>>> and I cannot see a NSP option in the protein prophet parameters. Does this >>>> mean that NSP is not being used when I run protein prophet? ... (i.e. am I >>>> ok to leave NSP on in iProphet?) >>>> >>>> >>>> Thanks >>>> Pete >>>> >>>> >>>> >>>> >>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "spctools-discuss" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to spctools-discu...@googlegroups.com. >>>> To post to this group, send email to spctools...@googlegroups.com. >>>> Visit this group at https://groups.google.com/group/spctools-discuss. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "spctools-discuss" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to spctools-discu...@googlegroups.com <javascript:>. >> To post to this group, send email to spctools...@googlegroups.com >> <javascript:>. >> Visit this group at https://groups.google.com/group/spctools-discuss. >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "spctools-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to spctools-discuss+unsubscr...@googlegroups.com. To post to this group, send email to spctools-discuss@googlegroups.com. Visit this group at https://groups.google.com/group/spctools-discuss. For more options, visit https://groups.google.com/d/optout.
