Incognito, Now you know I'm wrong. I predicted May 5 and the whole world wakes up to a different date.
If *we* win (remember, team submission) I will be glad to give my partner credit, which is rightfully hers :-) On May 4, 10:33 am, Incognito <[EMAIL PROTECTED]> wrote: > Wow! That is more information than I hoped for before going to sleep. > Thanks a lot! > > Muthu, > > I cannot believe you were right. I'm really starting to believe that > you have friends at Google. Once again, if you win, I'll be highly > suspicious! :) > Good luck and congratulations to those people that complained a lot > about not seeing server hits. According to Dan some (not all) have > done quite good. It appears that this Monday we may actually know who > the top 50 most promising developers from around the world are. Wow, I > can hardly wait for this Monday to come! > > On May 4, 12:22 am, "Dan Morrill" <[EMAIL PROTECTED]> wrote: > > > Ahh -- we've not been rigorous in consistently naming these various rounds > > and phases. Let me try and adopt that terminology for this thread, and > > explain again. > > ADC 1 == this $5,000,000 prize event going on now. > > ADC 2 == the second $5,000,000 prize event that will begin later this year. > > ADC 1 Round 1 == open participation with the deadline of 14 April, with 50 > > winners > > ADC 1 Round 2 == participation limited to the winners of ADC 1 Round 1, with > > 20 "final" winners > > ADC 1 Round 1 Phase 1 == reducing the original set of 1,788 submissions to > > 100 finalists > > ADC 1 Round 1 Phase 2 == picking the 50 ADC 1 Round 1 winners from the 100 > > finalists > > > Okay, phew. :) With those definitions, here is where we are: > > > - We sent out the submissions to judging a few days after the submission > > deadline of 14 April, and judging began. > > - Our 100 or so judges received the judging guidelines we provided, > > reviewed their assigned submissions, and reported data back to us. > > - Late last week, we applied our outlier mitigation techniques, > > identified the top 100 results, and sent them on to the final, separate > > panel of 15 or so judges to score and produce the final 50 ADC 1 Round 1 > > award recipients. > > > So in other words, we are currently in ADC 1 Round 1 Phase 2 as defined > > above. Once data from the judges comes in, we will notify the 50 award > > recipients and ADC 1 Round 2 will begin. > > > It has not escaped my notice even on vacation that there have been a number > > of discussions on server hits and so on. Obviously we don't have access to > > everyone's server logs, and we can't monitor what the judges have actually > > been doing (nor would we snoop if we could, since that seems really > > sketchy.) We've tried to automate everything we possibly can about the > > judging process, but the one thing we can't automate is the actual act of > > assigning scores, since that requires a human's brain. > > > The judges were given fairly detailed guidance on how to calibrate their > > scores, and what to review. For instance, they are aware that they are > > supposed to read documentation and do their best to test all the features. > > In the end, though, each judge is going to test to his or her own > > satisfaction. I'm not sure how reliable it is to correlate judge reviews > > with observed server hits. Some apps might have sporadic bugs that prevent > > network accesses. Some judges may have decided they didn't need to see a > > particular feature. And before you cry foul, know that some people who have > > inquired about "missing" server hits have actually done quite well. Judges > > are just as likely to say "this is cool, I don't need to see any more" as > > they are to say "this is so uncool, I don't need to see any more." On the > > whole, our judges have been excited to participate, and I expect that they > > are being as conscientious as they can be. > > > The one thing I can tell you with certainty is that I have answered quite a > > few private inquiries, and in all but one case the judges responded with > > legitimate scores, rather than scores that say something went wrong or the > > review was incomplete. Our only data points are what the judges give us, > > because that's the only factor we can't automate. Since the judges are > > telling us that they reviewed to their satisfaction, we can only take their > > word for it. > > > We've tried really hard to make sure that the only thing that affects > > scoring is what you put in front of the judges. But the entire goal of the > > ADC is to leverage plain old human judgment. > > > - Dan > > P.S. - watch for gory details on the nuts & bolts of all this in the near > > future. > > > On Sat, May 3, 2008 at 1:58 PM, Finn Kennedy <[EMAIL PROTECTED]> wrote: > > > Dan, > > > > thank you for the responses. A couple of follow ups. > > > > With Phase 2 I meant the 100 being winnowed down to 50. From the ADC > > > Judging Process page: > > > > "In Phase 2, the 100 highest-scoring submissions will be all be sent to a > > > new panel of judges (which may or may not include one or more of the > > > judges > > > who participated in Phase I judging). > > > ... > > > The 50 entries with the highest scores in Phase 2 judging will move on to > > > Round 2 of the Challenge..." > > > > Just for clarification, are there again groups of judges assigned to look > > > at a subset of the top 100 entries? Or are the entire set of entries > > > judged > > > by all the judges in Phase 2? > > > > Is the "outlier" procedure still used for Phase 2? By outlier I mean the > > > review of scores not matching the rest of the scores for the application > > > (mentioned on the board). > > > > I totally understand the want to have a fair playing field. It would not > > > be fair to extend advantages to winners that can make the trip to Google > > > I/O. > > > > Finn > > > > On Sat, May 3, 2008 at 3:27 PM, Dan Morrill <[EMAIL PROTECTED]> wrote: > > > >> Well, for all intents and purposes, Phase II begins as soon as we > > >> announce > > >> the 50 Phase I winners. It's not like we could stop the winners from > > >> starting right away, anyway. :) It looks like we are still on track to > > >> announce those winners next week. > > >> A different set of judges is reviewing the 100 applications. The top 100 > > >> applications are "reset" and rejudged from scratch by a different group > > >> of > > >> judges, who have no knowledge of the previous judges' scores. > > > >> We're thinking about ways to work with the 50 Phase I winners, but that > > >> might not necessarily include anything formal at Google I/O. (We don't > > >> want > > >> to require anyone to attend, and we don't want to give any of them an > > >> unfair > > >> advantage.) > > > >> - Dan > > > >> On Fri, May 2, 2008 at 6:58 PM, finnk <[EMAIL PROTECTED]> wrote: > > > >>> Having reread the Judging Process (http://code.google.com/android/ > > >>> adc_judging.html <http://code.google.com/android/adc_judging.html>), I > > >>> have a couple of questions. > > > >>> Since we are coming up on the week of May 5th, when is Phase II > > >>> starting? > > > >>> Will the entire panel of judges review the whole set of 100 > > >>> applications, or is the set of 100 split into groups and distributed > > >>> randomly again? > > > >>> Are there any differences between Phase I and Phase II? > > > >>> On an slightly related note, is there anything planned for the top 50 > > >>> planned at Google IO? > > > >>> Also, for Google IO: If you are traveling from Austin, TX, there are > > >>> direct flights from Austin to San Jose International. You can then > > >>> take CalTrain (http://www.caltrain.com) from close to the airport to > > >>> the San Francisco stop. It is on 4th, same street as the Moscone > > >>> Center. > > > >>> Of course I am not a travel agent/planner, so please double check > > >>> everything yourself. > > > >>> Finn- Hide quoted text - > > > - Show quoted text - --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Android Challenge" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/android-challenge?hl=en -~----------~----~----~----~------~----~------~--~---
