Ludovic, Go ahead and post the files to the newsgroup's file area (http://groups.google.com/group/spctools-discuss/files), and hopefully one of the validation experts will take a look.
I will point out that PeptideProphet uses random initialization for it's curve fitting (EM algorithm). So it's not out of the question that you'd see some small differences between runs on the same data files, regardless of the order. Can you provide some measure of the differences between runs for the reordered datasets? -Natalie On Tue, May 18, 2010 at 4:35 AM, lgillet <[email protected]> wrote: > Hi everybody, > I recently encountered a "bug" I think when people in my lab installed > the newest TPP (v4.3 JETSTREAM rev 1, Build 201004201202 (linux)), > especially when I try to confront the result to v4.0 which was our > former "benchmark" version. > When searching the same 4 pep.xml files with v4.0 and v4.3, I get an > incredible difference in decoy hits number. For example, with v4.0, > p>0.9, I would get my "regular" 1% decoy, while with v4.3, p>0.9, I > get above 25% of decoys?!?? > All the interact are run with the following options: xinteract -OApld - > ddecoy *.pep.xml > I could nail down the "problem" to the PeptideProphetParser which > behaves very differently between v4.0 and v4.3, while InteractParser > (which introduces the "is_rejected=1" tags) and RefreshParser do not > influence the results. > But at the moment, I do not know if it is an issue of the decoy > statistical distribution of prophet or not... > > One more thing that makes me even more suspicious is the fact that, > only with TPP version 4.3, if you search those files in a difference > order (let say: xinteract file1 file2 file3 Vs xinteract file3 file2 > file1), you do get differences in the results as well?!? > > I am willing to send the 4 pepxml where those observations are the > most critical to David or Luis or anybody interested, but I truly > believe that there might be something going wrong with the TPP v4.3. > > Let me know to whom I should post the files. > > Best regards, > > Ludovic > > -- > You received this message because you are subscribed to the Google Groups > "spctools-discuss" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/spctools-discuss?hl=en. > > -- You received this message because you are subscribed to the Google Groups "spctools-discuss" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/spctools-discuss?hl=en.
