Dan, Thank you for the detailed answer. This process definitely avoids subjective judging decisions.
On Apr 18, 11:14 am, "Dan Morrill" <[EMAIL PROTECTED]> wrote: > This is correct. We're providing rather detailed judging guidelines > intended to help judges calibrate their scores. We also plan to > post-process the results looking for outliers (such as cases where one judge > gave a score that varies significantly from the others') to help make sure > the results are fair. > > Panel size will be at least 3, depending on how many judges we have; > currently the panel size is expected to be 4. > > - Dan > > > > On Fri, Apr 18, 2008 at 7:38 AM, Living Sword <[EMAIL PROTECTED]> wrote: > > > i dont think apps will be judged relatively. Each app will have its > > own scores based on the 4 criteria. Then apps with highest scores will > > move on. > > > On Apr 18, 6:42 pm, viktor <[EMAIL PROTECTED]> wrote: > > > I hope that each submission will be valuated by a panel of judges, not > > > an individual judge:http://code.google.com/android/adc_judging.html > > > > To be objective, a panel must consist of at least 3 judges. > > > > Let's assume that each of 100 judges receives 17.8 submissions. A > > > judge will score an app relatively to other apps in the same "bucket". > > > In most cases, this will lead to one app from a bucket that makes into > > > the first 100. Apps are randomly packed into buckets. This may result > > > in "strong" (several good apps) and "weak" (no good apps) buckets. A > > > second app in a strong bucket can be significantly better than the > > > best app in a weak bucket. But this second app may not make the first > > > 100 if buckets are reviewed by individual judges because cross-bucket > > > judging will not be executed. > > > > So, larger buckets lead to fairer judging. > > > > Dan Morrill, please let us know the size of a bucket when you have a > > > free moment. > > > > > - Show quoted text -- Hide quoted text - > > - Show quoted text - --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Android Challenge" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/android-challenge?hl=en -~----------~----~----~----~------~----~------~--~---
