Dan, Thank you very much for this insight into the judging process. It is very relieving to know that you try your best to make the judging process as fair as possible.
Peli On Apr 18, 5:14 pm, "Dan Morrill" <[EMAIL PROTECTED]> wrote: > This is correct. We're providing rather detailed judging guidelines > intended to help judges calibrate their scores. We also plan to > post-process the results looking for outliers (such as cases where one judge > gave a score that varies significantly from the others') to help make sure > the results are fair. > > Panel size will be at least 3, depending on how many judges we have; > currently the panel size is expected to be 4. > > - Dan > > On Fri, Apr 18, 2008 at 7:38 AM, Living Sword <[EMAIL PROTECTED]> wrote: > > > i dont think apps will be judged relatively. Each app will have its > > own scores based on the 4 criteria. Then apps with highest scores will > > move on. > > > On Apr 18, 6:42 pm, viktor <[EMAIL PROTECTED]> wrote: > > > I hope that each submission will be valuated by a panel of judges, not > > > an individual judge:http://code.google.com/android/adc_judging.html > > > > To be objective, a panel must consist of at least 3 judges. > > > > Let's assume that each of 100 judges receives 17.8 submissions. A > > > judge will score an app relatively to other apps in the same "bucket". > > > In most cases, this will lead to one app from a bucket that makes into > > > the first 100. Apps are randomly packed into buckets. This may result > > > in "strong" (several good apps) and "weak" (no good apps) buckets. A > > > second app in a strong bucket can be significantly better than the > > > best app in a weak bucket. But this second app may not make the first > > > 100 if buckets are reviewed by individual judges because cross-bucket > > > judging will not be executed. > > > > So, larger buckets lead to fairer judging. > > > > Dan Morrill, please let us know the size of a bucket when you have a > > > free moment. > > > > > - Show quoted text - --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Android Challenge" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/android-challenge?hl=en -~----------~----~----~----~------~----~------~--~---
