Hi David, TPP is installed in different servers in our Institute. I have re- uploaded a new file (lgillet_interact-again.zip) for which the TPP xinteract was performed on the same server and with different versions of the TPP. You can see that the results are still very different, even the scrambled case. Note that I used the version TPP v4.3 JETSTREAM rev 1, Build 201004201202 (linux); which I do not know if it is the same as the SVN TPP that you mentioned or a "nightly-built". - Can you re-confirm my results using your installation of TPP with my 4 pep.xml files? - Can you re-confirm the differences in decoy % using your installation of TPP between TPP V4.0 and V4.3 with my 4 pep.xml files? Thanks again, Ludovic
On May 26, 9:01 pm, David Shteynberg <[email protected]> wrote: > Hi Ludovic, > > I was unable to duplicate the different results on different order of > input using the latest version of SVN tpp or version 4.3.1. I noticed > that your two analyses point to different locations. Are you sure > that the files at these locations are identical? > > Thanks, > -David > > On Wed, May 26, 2010 at 10:47 AM, lgillet <[email protected]> wrote: > > Hi David, > > > all my apologizes, the rar file got corrupted probably during the > > upload (the original on my HD was fine). > > I have uploaded again a zip file this time: lgillet_pepxml-again2.zip > > I hope that works this time (after download, I can decompress it > > back). > > Thanks for having a look at this issue. > > Best, > > Ludovic > > > On May 25, 7:24 pm, David Shteynberg <[email protected]> > > wrote: > >> Hi Ludovic, > > >> It seems the file you uploaded lgillet_pepxml_for_TPP4.3.rar is > >> corrupted. At least I am unable to open it. Please upload again. > > >> Thanks, > >> -David > > >> On Wed, May 19, 2010 at 2:54 AM, lgillet <[email protected]> wrote: > >> > Hi David, Hi Natalie, > > >> > I just posted the 4 pepxml files which give me the most striking > >> > differences in results between TPP-V4.0 and TPP-V4.3: > >> > lgillet_pepxml_for_TPP4.3.rar. I also posted the results > >> > (interact.pep.xml) which I obtain from running TPP-V4.0, TPP-V4.3 and > >> > TPP-V4.3 on scrambled file order (file #4>#3>#2>#1): lgillet_interact- > >> > results.rar. > >> > I really tried my best to figure out what the problem could be. > >> > Maybe you could re-run the same analyses (TPP-V4.0, TPP-V4.3, TPP-V4.3 > >> > with the scrambled file order) and let me know if you confirm my > >> > results or if there is something wrong maybe with the compiled version > >> > we have on our server (could still be a possibility). > >> > Finally, to answer Natalie's question, the differences are quite > >> > dramatic (to my opinion) between V4.0 and V4.3 (I would not have > >> > worried about 1-2% differences in IDs), but here, I am passing from 1% > >> > decoy (V4.0) to 23% decoy (V4.3) hits (at the same proba > 0.9). Also > >> > the number of unique peptides reported by V4.0 and V4.3 is quite > >> > different (2150 and 3161 resp.). Finally, many decoy hits pulled up in > >> > V4.3 with a prob>0.9 have actually a very bad MS/MS spectrum and a > >> > very low prob<0.01 (only reported if you use -p0 option) on V4.0. > > >> > Have a look at those MS/MS spectra for example: > > >> > 20100422_04_control_07.c.07700.07700.4 > >> > 20100422_04_control_07.c.02864.02864.3 > > >> > Let me know if you need any extra information. > > >> > Thanks a lot for your help on that. > > >> > Best, > > >> > Ludovic > > >> > On May 18, 11:21 pm, Natalie Tasman <[email protected]> > >> > wrote: > >> >> Ludovic, > > >> >> Go ahead and post the files to the newsgroup's file area > >> >> (http://groups.google.com/group/spctools-discuss/files), and hopefully > >> >> one of the validation experts will take a look. > > >> >> I will point out that PeptideProphet uses random initialization for > >> >> it's curve fitting (EM algorithm). So it's not out of the question > >> >> that you'd see some small differences between runs on the same data > >> >> files, regardless of the order. Can you provide some measure of the > >> >> differences between runs for the reordered datasets? > > >> >> -Natalie > > >> >> On Tue, May 18, 2010 at 4:35 AM, lgillet <[email protected]> > >> >> wrote: > >> >> > Hi everybody, > >> >> > I recently encountered a "bug" I think when people in my lab installed > >> >> > the newest TPP (v4.3 JETSTREAM rev 1, Build 201004201202 (linux)), > >> >> > especially when I try to confront the result to v4.0 which was our > >> >> > former "benchmark" version. > >> >> > When searching the same 4 pep.xml files with v4.0 and v4.3, I get an > >> >> > incredible difference in decoy hits number. For example, with v4.0, > >> >> > p>0.9, I would get my "regular" 1% decoy, while with v4.3, p>0.9, I > >> >> > get above 25% of decoys?!?? > >> >> > All the interact are run with the following options: xinteract -OApld > >> >> > - > >> >> > ddecoy *.pep.xml > >> >> > I could nail down the "problem" to the PeptideProphetParser which > >> >> > behaves very differently between v4.0 and v4.3, while InteractParser > >> >> > (which introduces the "is_rejected=1" tags) and RefreshParser do not > >> >> > influence the results. > >> >> > But at the moment, I do not know if it is an issue of the decoy > >> >> > statistical distribution of prophet or not... > > >> >> > One more thing that makes me even more suspicious is the fact that, > >> >> > only with TPP version 4.3, if you search those files in a difference > >> >> > order (let say: xinteract file1 file2 file3 Vs xinteract file3 file2 > >> >> > file1), you do get differences in the results as well?!? > > >> >> > I am willing to send the 4 pepxml where those observations are the > >> >> > most critical to David or Luis or anybody interested, but I truly > >> >> > believe that there might be something going wrong with the TPP v4.3. > > >> >> > Let me know to whom I should post the files. > > >> >> > Best regards, > > >> >> > Ludovic > > >> >> > -- > >> >> > You received this message because you are subscribed to the Google > >> >> > Groups "spctools-discuss" group. > >> >> > To post to this group, send email to > >> >> > [email protected]. > >> >> > To unsubscribe from this group, send email to > >> >> > [email protected]. > >> >> > For more options, visit this group > >> >> > athttp://groups.google.com/group/spctools-discuss?hl=en. > > >> >> -- > >> >> You received this message because you are subscribed to the Google > >> >> Groups "spctools-discuss" group. > >> >> To post to this group, send email to [email protected]. > >> >> To unsubscribe from this group, send email to > >> >> [email protected]. > >> >> For more options, visit this group > >> >> athttp://groups.google.com/group/spctools-discuss?hl=en. > > >> > -- > >> > You received this message because you are subscribed to the Google > >> > Groups "spctools-discuss" group. > >> > To post to this group, send email to [email protected]. > >> > To unsubscribe from this group, send email to > >> > [email protected]. > >> > For more options, visit this group > >> > athttp://groups.google.com/group/spctools-discuss?hl=en. > > >> -- > >> You received this message because you are subscribed to the Google Groups > >> "spctools-discuss" group. > >> To post to this group, send email to [email protected]. > >> To unsubscribe from this group, send email to > >> [email protected]. > >> For more options, visit this group > >> athttp://groups.google.com/group/spctools-discuss?hl=en. > > > -- > > You received this message because you are subscribed to the Google Groups > > "spctools-discuss" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]. > > For more options, visit this group > > athttp://groups.google.com/group/spctools-discuss?hl=en. -- You received this message because you are subscribed to the Google Groups "spctools-discuss" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/spctools-discuss?hl=en.
