In the context of your problem as phrased, I'd suggest you look at sampling plans & inspection sequences.
You have an inspection (a test procedure) that results in identification, P1 exists, or P1 does not exist. You have an alpha risk and a beta risk. (the % incorrectly identified P1 may = alpha + beta, depending on how you define things). If I inspect a lot of material, I can have an inspection sequence where I sample n pieces, determining in each case with absolute certainty that condition P1 exists in any given piece. for a given sample size, n, I can calculate an OC (operating characteristic) curve, from which I determine the likelihood of establishing the acceptability or reject ability of the entire lot. Or I could use the MIL standards (now with a new ISO name) to specify the minimum sample size and number of P1 detects. I think a strong cognate analytical structure could be built. If so, then sampling plans will solve your problem. I suppose you could also report % correct ID (subject to the alpha + beta issue above), and compare two methods using a t test using proportions. I feel less comfortable about this, if you do not examine alpha & beta risks separately. Jay JGR wrote: > Hi, > > I have the following problem: > I need to select between test procedures T1 and T2. Previous > observations show that T1 correctly identified problem P1 in 29 out of > 30 cases and T2 correctly identified problem P1 in 5 out of 5 cases. I > want to have a test procedure selection measure, which represents both > accuracy of the procedure and frequency of previous observations. > > Thanks, > JGR > . > . > ================================================================= > Instructions for joining and leaving this list, remarks about the > problem of INAPPROPRIATE MESSAGES, and archives are available at: > . http://jse.stat.ncsu.edu/ . > ================================================================= -- Jay Warner Principal Scientist Warner Consulting, Inc. 4444 North Green Bay Road Racine, WI 53404-1216 USA Ph: (262) 634-9100 FAX: (262) 681-1133 email: [EMAIL PROTECTED] web: http://www.a2q.com The A2Q Method (tm) -- What do you want to improve today? . . ================================================================= Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at: . http://jse.stat.ncsu.edu/ . =================================================================
