> Well... "valuable" w.r.t. measurement usually implies that you know > what the question is. Collecting data without knowing the question is > a problem. The spirit of your questions seem to be "what is the > activity/pulse of this community?" If that's the question, then that > might make it easier to think of what to measure.
I think that's accurate - "is the TOS community alive and helping professors teach open source classes?" Max and I pow-wowed for a bit after lunch today and I decided I was going to limit myself to measuring 5 things, every week: 1. TOS presence at events - that's http://mchua.fedorapeople.org/tos-scorecard/ambassadors.png, except expanding it to be all TOS community members rather than just students. 2. TOS publications - that's http://mchua.fedorapeople.org/tos-scorecard/publications.png, basically any paper any TOS community member writes about TOS. 3. TOS Planet activity - # of total posts, # of individuals posting, # of individual classes represented by these posts. 4. TOS list activity - # of total messages, # of individuals posting, # of subscribers (which I don't think I have permission to view, actually - this is something the infra team would have to decide to give me access to, else I'll drop that metric). 5. POSSE module attendance - how many folks show up each week to the online classes we're teaching? (Just going for sheer numbers right now; once we get a gauge on what types of folks show up - profs, students, devs, etc - we might also start tracking proportions.) Lots of open questions about how precisely to get accurate numberss on these; need criteria, etc and there are some things to address like "boy, it's hard to count Planet posts when Planet is down!" but it gives me a solid target to aim for. I'll be running these numbers for the first time for this week's stats, and we'll see how hard they are to generate. > If you genuinely thought that the study of FOSS communities was going > to play a role in your dissertation work, then a dashboard-like > project could be a very powerful way to get research data. However, > you can likely pursue a dissertation with three case studies just as > well as a firehose of incomprehensible commit logs... it all depends > on your question. But, in the meantime, if you designed an amazing, > best-of-breed, FOSS participation dashboard... yeah, I'd use it. :D I'm hoping that I'll work on this stuff and a few years from now look up and go "...wait, 70% of my dissertation data has already been collected by shell scripts over the past 3 years, I just need to do a bunch of analysis and writing!" I'd love to have a ton of quantitative data to get some general trends across classes and institutions and help us zone in on more interesting places to look for case studies... but that's wishful thinking from someone who's not yet learned to do proper research and does not have a properly-formed research question, and I realize all this is potentially a ton of work and I need to figure out a smallish bite-sized bit that will actually let me graduate before I retire. I'll start very small, with metrics that I personally care about and can manually capture in a very short amount of time every week, then see how that goes before I even think about the automation. Not getting all starry-eyed about a groundbreaking new research platform right now or anything... just trying to consistently measure *something* and see if it gets any insights, and then going from there. --Mel _______________________________________________ tos mailing list tos@teachingopensource.org http://teachingopensource.org/mailman/listinfo/tos
