Hi Ali, my thoughts are that for this dataset and analysis, PeptideProphet+iProphet (PP+iP) are discriminating between correct and incorrect much better than the other analysis. The vast majority of peptides are deemed by PP+iP as very likely to be correct or very likely incorrect with a rather small population of IDs at intermediate confidence. The other analysis seems to be unsure about many more. In fact the curve is extremely steep indicating not very good discriminating power between correct and incorrect. There are many more in the intermediate range.
The PP+iP analysis looks quite similar to what we are used to for modern datasets. The steepness of the other analysis seems a bit worrying to me. How are the PEPs calculated? Can you perform an independent analysis of the PEPs to see if they agree with decoys or some other measure of errors? I would say that in the range that matters (FDR < 1%) PP+iP is giving you a better result. It is puzzling that the total number of correct hits at FDR 2.5% is so much lower fir PP+iP. I couldn’t say from the data shown why that is. Maybe the PEPs are not correct or MQ is finding more IDs because of different search parameters? I don’t know. Regards, Eric *From:* Ali [mailto:[email protected]] *Sent:* Wednesday, January 04, 2017 1:06 PM *To:* spctools-discuss *Cc:* [email protected]; [email protected] *Subject:* Re: [spctools-discuss] PeptideProphet/iProphet Probability Distribution Dear Eric and David By accepting lower thresholds I actually mean accepting higher FDRs but that is not so useful here because there is not so many hits after the very high probabilities and I will not be gaining much by using lower thresholds. PeptideProphet gives me a lot of PSMs with very high confidence and very low FDR. But as I allow higher FDRs like 2-3% it falls behind PEP from MaxQuant. Here is another plot to better show my point. I have compared the combined results above with Max Quant's results as I increase the FDR. As it can be seen, number of unique peptides is more with PepPro/iPro for low FDR, but in the end it falls behind PEP. <https://lh3.googleusercontent.com/-VZgUOiAvvno/WG1ggpOR3uI/AAAAAAAABBA/nyrjsBbLrGQ60eo3iqB3beMOVs4ZCXCAACLcB/s1600/Picture1.png> And this can be understood from the following plot: <https://lh3.googleusercontent.com/-jmsbfmBfbXA/WG1hNLP8e1I/AAAAAAAABBI/SBnRc5-naTw09vjKk0K4m9xcCBgS-1Z8QCLcB/s1600/Picture2.png> Do you have any thoughts on this? Thank you very much, Ali On Wednesday, January 4, 2017 at 3:08:14 PM UTC-5, Ali wrote: Dear David and Eric Thanks for your replies and suggestions. I thought that I might be able to increase identification by accepting hits with lower thresholds. I do understand what you mentioned here. Thanks again, Ali On Tuesday, January 3, 2017 at 1:57:38 PM UTC-5, Eric Deutsch wrote: In the ideal case where the PeptideProphet and iProphet had perfect discriminating power, i.e. the correct and incorrect PSM distributions do not overlap, you would expect a large peak at P=0 and P=1 and nothing in between. As a dataset diverges from this ideal and the correct and incorrect distributions begin to overlap, you will still have very large peaks near P=0 and near P=1, with a small number of intermediate values where the probability is between 0 and 1. So, in short, your distribution is exactly what you would expect based on what you did. It is indicative of a great dataset. You should follow David’s suggestion, and then if you replot, you will see huge peaks at 0 and 1 with rather little in between. Regards, Eric *From:* [email protected] [mailto:[email protected]] *On Behalf Of *David Shteynberg *Sent:* Tuesday, January 03, 2017 10:36 AM *To:* spctools-discuss *Subject:* Re: [spctools-discuss] PeptideProphet/iProphet Probability Distribution These results are showing you that iProphet has great discriminating power even when you restrict the input to only non-low probability PSMs (filtering with PeptideProphet probability). The best way to run iProphet is to give it all the data from PeptideProphet (use a PeptideProphet minimum probability of 0). -David On Mon, Jan 2, 2017 at 9:26 PM, Ali <[email protected]> wrote: Dear all I am plotting the distribution of the assigned PeptideProphet and iProphet probabilities to my search results. I am seeing a rather strange pattern shown below. Why am I seeing so many hits with very high confidence (probability >0.98) but suddenly after that threshold the number of assigned probabilities to the PSMs decrease abruptly? What could be the reason for this pattern? One would expect to see all ranges of probabilities for PSMs. <https://lh3.googleusercontent.com/-BeikGpozjkw/WGsygFV83UI/AAAAAAAABAg/DcG6lRxucM4IRclvdadal-eFSaA05sd6QCLcB/s1600/Picture1.png> This figure is the combination of different search engines: - Comet (min PepPro=0.5) - X! Tandem GPM (min PepPro=0.05) - X! Tandem TPP (min PepPro=0.05) The horizontal axis is between 0.5 and 1. and I can see the same pattern for the PeptideProphet distribution of each one of these search engine results including SpectraST (Except for X! Tandem GPM). Thank you very much, Ali -- You received this message because you are subscribed to the Google Groups "spctools-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/spctools-discuss. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "spctools-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/spctools-discuss. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "spctools-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/spctools-discuss. For more options, visit https://groups.google.com/d/optout.
