Decisions: * Neither Sugar Labs, OLPC or any other distribution can take responsibility over *all* Activity testing. Only sanity checking (and needed testing) on the subset they've decided to be responsible for. We should work together to encourage Activity maintainers to test their own Activities, provide them the necessary tools, policies and information. * During these coordination meetings and a few email threads, we made huge progress on upstream/downstream responsibility division on testing. We are all comfortable to move ahead with the plan now. Yay!
Actions: * Marco to run point on "Activity testing infrastructure" at Sugar Labs. * Marco to keep championing activity infrastructure and maintenance work flows at Sugar Labs. * Marco to schedule a meeting about automated testing, probably in the context of development team meetings. * Mel to communicate with OLPC management and internal testing group about the outcome of these meetings. * Mel to sleep at night. She was falling asleep in the middle of the conversation and almost derailed Marco plans to finish in one hour. Lessons learned: * Always write your minutes just after the meeting, or you will end up wasting time to reread the whole log. (if you have a flaky memory like mine, at least). * Make sure that everyone take actions, Simon has apparently been lazy this time! Thanks: * To Simon and Mel for the entertaining and productive conversation. I'm excited that we removed pretty much all the road blockers. There will be more coordination to do, I'm sure, but we can start running now! Full log: <marcopg> ok I guess we can start the next meeting now I'm supposed to run it, but I don't know how to run meetings but it's only a few of us, so it won't be too bad! so activities the question last time was... who tests activities? marcopg mchua_ mungewell1 meeting m_stone mtd morgs mchua_: is the olpc community testing group going to do it? <mchua_> "Not OLPC," says the Mel. "Only when we want to verify Activities we're shipping on the default image for XOs," says the Mel. <marcopg> mchua_: OLPC as org or as community too? <mchua_> marcopg: as org, and as far as the org will support community testing, at least for now. marcopg: there's a reason I started off community testers with g1g1-2008 activities - it's a really easy way to argue that this, clearly, is something OLPC Should Have Tested, as we Ship It. <marcopg> mchua_: so OLPC strategy is to rely on upstream worth activities quality? mchua_: that's what I think too, if you ship something you need to test it from the upstream point of view <mchua_> marcopg: Well, it's tricky. We definitely don't *support* Activities, even the few we ship. <marcopg> mchua_: I understand, but I've not been able to make sense out of that yet <mchua_> marcopg: and with limited resources, I would prioritize OLPC testing towards XO-specific code (like our build, say.) marcopg: hm, then I should try to clarify... is there any particular point you had questions on? marcopg: or should I start trying to explain my current understanding of it? <marcopg> mchua_: prioritizing make sense clearly... but I don't see how you can not support something you ship I guess what I'm unclear on is how does it make sense to ship a product, and *not* support a very relevant part of it? think to the extreme case if no activities would work what good would it be if the OS/core Sugar was perfect? <mchua_> Oh, I agree that without Activities, a perfect ship of Sugar/XO-OS would be useless. <marcopg> mchua_: but you disagree that OLPC should ensure that Activities are working well? * erikos can not kick the bot - just tried <mchua_> Maybe a similar case would be... I'm not sure how Red Hat support works, but if someone writes $random_third_party_application, Red Hat doesn't necessarily support it automagically, right? <marcopg> mchua_: totally agreed about random third parties mchua_: but making every activity a random third party seems to push it too far <mchua_> marcopg: I disagree that it is the responsibility of OLPC to ensure all Sugar Activities are working well. I agree it is the responsibility of OLPC - a responsibility towards its customers - to ensure a small subset of Sugar Activities they're likely to care about are working well. <marcopg> mchua_: then we completely agree :) <mchua_> marcopg: I think that this responsibility can best be carried out by making sure that a non-OLPC test/dev group for that Activity (which may include OLPC employees) makes sure that Activity works well, <marcopg> so OLPC, through his community or not, should make sure that this small subset is tested, right? <mchua_> since it's likely for Activities we ship (and therefore have a responsibility to sanity-check) to change from build to biuld. <mchua_> marcopg: Yep. --- andresambrois is now known as aa <mchua_> marcopg: Kind of like how OLPC QA has a vested interest in having SL's BugSquad triage awesomely, even though SL's BugSquad is Not Us. (In fact, we have a huge incentive to keep the two separate.) <marcopg> mchua_: make sure that the activity works well on the XO (I guess) so this is still some kind of downstream testing, prolly mchua_: did you already start to work on this? or is it just a plan? <mchua_> marcopg: Yep. But OLPC QA won't really care if the Activity works well in, say, Debian. And probably can't justify resources spent to do that. Just on the stuff we ship, and on whether it works on the XO. <marcopg> mchua_: sure, that's why I call it downstream testing <mchua_> marcopg: Well, the G1G1-2008 Activity testing is my first attempt at that limited sphere of responsibility (and it's far from perfect, but better than the almost-nothing we had before.) <marcopg> mchua_: my feeling is that this kind of testing should *not* go through SL basically <mchua_> marcopg: I don't want that to be interpreted as "OLPC will do all Activity testing, ever!" though - and want to work with SL to find ways to encourage Other People to do it. <mchua_> it'll make both of our lives easier. <mchua_> marcopg: I agree it shouldn't be a SL thing either. <marcopg> mchua_: and that from the upstream point of view it should probably work just like core testing i.e. we get and triage the bug reports (and fix them hopefully!) <erikos> agreed here <marcopg> OK this is really cool mchua_: thanks a lot, I was really struggling to understand olpc position on activities! <mchua_> marcopg: well, wait... is Activity development (making sure Activity bugs get fixed) the responsibility of core-sugar devs? <mchua_> basically, "Who's Responsible For Activities?" <marcopg> mchua_: nope, of the activity upstreams well not exactly so <mchua_> Is it SL? The Activity maintainer (as an independent project?) <marcopg> we have a group of activities which are part of the SL release process <mchua_> It's definitely not OLPC, except for the "we need to check what we ship" bit. * mchua_ listens <marcopg> the group is called Fructose I'm not completely sure it's the right setup and I'm willing to question that at some point but, as long as, things are setup this way I think SL as a community is responsible for the Fructose activities and only for those <erikos> development and triaging wise <mchua_> marcopg: in the same way OLPC is responsible for the Activities we ship in signed images? <marcopg> for the other activities we basically provide services <erikos> mchua_: yup <marcopg> mchua_: not quite mchua_: in the same way we are responsible for the core <erikos> mchua_: they are officially part of the release <marcopg> SL doesn't have customers, so we are not responsible in the same sense as olpc <erikos> oh right <marcopg> we don't do support etc mchua_: is it clear how it's supposed to work? And I'd also like your opinion about how good is this setup, or how you'd see it work (we can't change it for 0.84 at this point, but we might reconsider down the way) my feeling lately is that this setup is too inflexible different deployments might want pretty different set of activities <erikos> marcopg: yeah - but we have only a very small core marcopg: and sugar without that core is nothing <marcopg> erikos: yup, but it would grow ideally <erikos> marcopg: the core? <mchua_> marcopg: but so would the maintenance you have to do <mchua_> (you == SL) <marcopg> mchua_: sure and that's part of the reason I'm starting to dislike it ;) <mchua_> So, as I understand it, what other open-source projects have done to solve this problem is to market it as "You've got to make your Activity good enough to include in our Release of Awesome." <marcopg> we might spend time maintaining things which would not be generally useful <mchua_> (which includes stuff like testing and docs.) <marcopg> mchua_: very correct and also the application should be somewhat generally useful and obviously not duplicate anything else in the core s/core/release <mchua_> Being chosen for Fructose, or the G1G1 image, is an *honor.* It's an honor we don't have as much transparency on (as in, "I wish my Activity could be included, what do I have to do?" does not exist.) <marcopg> mchua_: it does exist :) perhaps not well communicated though! <mchua_> marcopg: hm, I am missing many things then :) where do I find this? <marcopg> somewhere on the wiki :P (re not well communicated...) let me see mchua_: http://sugarlabs.org/go/DevelopmentTeam/Release#New_modules_proposal mchua_: I mailed the list about it a couple of times and we had a few proposals about it but well, yeah, the documentation is very poor etc mchua_: the fact that you had no idea about it, is obviously an indication that something is broken communication wise :( <mchua_> marcopg: oh! I mostly meant for OLPC (Greg Smith has done an admirable job of starting to collect proposals and make that process more transparent for 8.2.0, though). <marcopg> mchua_: oh I thought you was referring to SL. SL sucks at that anyway :) Greg has been doing better! <mchua_> marcopg: but, yeah, I think it applies to both SL and OLPC (although in slightly different degrees/ways). <mchua_> <3 greg smith. I don't know what we'd do without him. <marcopg> so the consensus here is that we should keep Fructose but be careful to not expand it too much? that's at least my understanding of what erikos and mchua_ was saying <erikos> sounds good with me <erikos> marcopg: as well - activities can be dropped or replaced in my opinion <marcopg> erikos: sure (though in practice removal is sort of difficult) <mchua_> from the perspective of OLPC, Fructose probably doesn't matter as much (only inasmuch as it overlaps with what we've chosen to ship, really. ;) but that sounds good to me. back to the original question of "who tests Activities?" - it sounds like neither SL nor OLPC can take responsibility over all Activity testing, only sanity checking (and needed testing) on the subset we've decided to be responsible for. so my question is what we can do to work together to encourage Activity maintainers to test their own Activities, to ease the load on both of us as "downstream." <marcopg> mchua_: the idea is that it will tend to overlap a lot with what you decided to ship, otherwise Fructose is going to be pretty useless <mchua_> ...until OLPC doesn't make up nearly all of SL's user base, but yeah. <marcopg> yeah that's a good question activity testing sprints sounds like a great thing, independently from who organize them <erikos> yup testing sprints sounds great <marcopg> stuff like SoaS will also get in quite a bit of testing on activities hopefully <mchua_> I think Activity testing infrastructure helps too; that's what I've been trying to set up from the OLPC side - I hope that what we've come up with is migrateable. <marcopg> maybe activity testing sprints should be co-organized by SL and OLPC (and by SoaS etc) <mchua_> marcopg and erikos, can SL host the Activity development and testing resources? <marcopg> a bunch of people testing the same activities on a bunch of different distros lots of fun! mchua_: I think so <mchua_> like their trac instances, and where we report bugs on them, and gitorious and such for them? (I know some of this is already in the works.) <erikos> mchua_: development ressources we do host <marcopg> mchua_: hosting at SL sounds like a good idea <erikos> mchua_: we just moved git <mchua_> Excellent. How can I move the current Activity testing resources upstream, then? Both in terms of infrastructure, but also process and the people who are doing the testing? <marcopg> can you enumerate the infra you need to move? or I can do it, if you like * mchua_ is trying to steer helpful resources in the SL-hat direction, and trying to make it as easy as possible to take and clone (and then we'll delete it here) <mchua_> marcopg: the move should take place after Dec. 25, when our current Activity test sprint ends. But basically, http://wiki.laptop.org/go/G1G1_Activity_testing and all the resources that it links to. <mchua_> (or things that serve the same purpose.) <marcopg> mchua_: wiki pages only? I'm planning to help moving dev resources here git and trac compoennts <mchua_> marcopg: also a trac infrastructure for reporting bugs, but it sounds like you already have that down marcopg: maybe a more coherent way of rephrasing it would be <mchua_> "if SL is hosting Activity development/testing as a service to the Activity development/user community, there are some things I think it should provide" <marcopg> mchua_: things == infrastructure? <mchua_> marcopg: yeah <mchua_> Some are things that are also dev resources, like code hosting (and an easy way for testers to download/install both latest-stable and bleeding-edge packages) and Trac (and procedures for reporting bugs, yay BugSquad!) <marcopg> oh I'm pretty sure we can get infra team to give us what we need :) <mchua_> Some are things that are a little bit more test specific, like... *searches brain for list* <marcopg> mchua_: I guess my question is how do we want to go about this then <mchua_> ...a way for each Activity to keep track of its own test cases, test plans, and test results. And a way for the test community (SL's BugSquad, but also downstream projects) to search for and look at information about Activity test results across multiple Activities. <marcopg> i.e. how do we get to the infra team stuff we require <mchua_> And a way of contacting maintainers so it's easier to escalate any issues. <erikos> hmm <mchua_> marcopg: Yeah... if this sounds like the arrangement we want to have, I'm super-happy to work out the tickets that we need to file for the infrastructure team, but I'm not sure if this is the arrangement that you want, too. <erikos> the bug communication happen over trac <marcopg> mchua_: I like it. Can you think of any reason I'd not want it? :) <mchua_> erikos: Yep. (Another reason to figure out the upstream/downstream magic trac plugin - chalk a +1 to my motivation-to-work-on-that list.) <erikos> testing plans - would be good to have upstream i agree mchua_: ;p <marcopg> erikos: and activity user/devel info <mchua_> marcopg: more stuff to think about and maintain (as infrastructure), mostly. <erikos> marcopg: yeah we just need to enhance the Modules page <marcopg> well, activities are my nightmare anyway <mchua_> marcopg: the setup costs are high. But I believe they'll be a well-placed investment (and eventually, a necessary one.) <marcopg> it's the biggest blocker for SL atm (ask caroline!) <mchua_> marcopg: how so? <marcopg> it's a pain to get them into distributions because they are poorly maintained... * erikos is interested as well <marcopg> it's not even clear how to contact the maintainers <erikos> oh yeah :/ <marcopg> maintainers don't know how to package stuff properly etc SL needs to start to work with activity maintainers at all levels packaging, testing, user documentation * tomeu1 adds a Activities category to amo <marcopg> which doesn't mean we should do the work <erikos> ok i think - lets fix the glucose one first <marcopg> but provide the infra, the policies, the info that activity authors will use to do a better work <erikos> marcopg: we could do a sprint as well <marcopg> and I think the idea of upstreaming a bunch of activity related infra pieces, fits perfectly into this <erikos> once we have the basic infrastructure up <mchua_> From the OLPC side, I'm going to try to push Activity testing resources upstream, and help the SL-hosted infrastructure get started. That'll help us concentrate on 8.2.1 and 9.1. Is someone going to run point on "Activity testing infrastructure?" <marcopg> "run point"? my english sucks <mchua_> oh! sorry, "take as an ongoing action item" <marcopg> (great about the OLPC side) mchua_: I can take it I want to have some discussion about it in tomorrow dev meeting <mchua_> marcopg: I'm not sure how common the phrase "be the point person for this" is <marcopg> mchua_: oh that I understand, it's the "run" thing which confused me ;) <mchua_> marcopg: Thanks. I'm too hosed with CTW and 8.2.1 to really concentrate on Activity testing beyond our G1G1-2008 run right now, but can probably jump back in more helpfully in January. <marcopg> oh wait Activit testing infra :) damn, I took the action without knowing what it was exactly :P <mchua_> marcopg: So yeah, please, please do ping me on that if you need a hand after... say... Jan. 5. I should have more of a life then. <marcopg> mchua_: OK, I will take it I suspect I'll have to bug you a lot, though :) <mchua_> marcopg: (If you're too hosed and need to back out, that's okay. I understand. We /can/ come back to this right before FUDCON.) <marcopg> mchua_: I will do my best :) <mchua_> marcopg: That's not a problem at all - I just need someone else to take responsibility for the remembering and pushing-this-forward right now, because I'm staggering under too many things to keep track of right now. :) marcopg: thanks, Marco. <marcopg> time running out! * marcopg think mchua_ is trying to make me a 2 h meetin got make fun of me <marcopg> (can't type) * mchua_ didn't know we set a time limit for this one :) <mchua_> I'll stay as long as it takes. <marcopg> ok it seem like we have a plan on activities, yay! <mchua_> (...well, if it runs over 4 hours, I might have to get dinner, but... yeah.) * marcopg wonder if mchua_ doesn't remember what I said about time of this meeting or she is just making fun of me <mchua_> marcopg: yeah, I'm feeling *much* better about Activity testing now, thanks. And I'll make sure this gets communicated to the OLPC testing group and community. marcopg: oh, are we doing an hour? ah! sorry! * mchua_ has... a brain of great scatteredness today <erikos> marcopg: other points? <marcopg> mchua_: hehe I had said this one was going to be *exactly* one hour, thought you was kidding me ;) yeah OLPC * marcopg leaves the floor to mchua_ about this <marcopg> we discussed a part of that already with activities I guess mchua_: I guess my question is... how do you think what we discussed in this two meetings will be in regard to OLPC mchua_: to OLPC org in particular mchua_: it's our first customer, so we have to care about it :) mchua_: kim in particular seemed to be worried about upstream/downstream separation of the bug tracker mchua_: do you think we are clear enough on how to coordinate, to be able to reassure them? mchua_: anything you would like us to do, to make OLPC life easier erikos: these OLPC people are evil, see now she runned away so that I can't close my meeting in time! * mchua_ is here! really! just thinking... <marcopg> mchua_: :)) <erikos> :) <marcopg> mchua_: sleep helps to think faster :P <mchua_> do you know what kim's concerns were, and/or would you like me to talk with her about those? <mchua_> marcopg: :P marcopg: ...says the person who was also up 'till 4. <marcopg> (you was up much later!) erikos: do you remember? we had a thread on techteam <mchua_> As far as I'm concerned, I'm completely confident that we'll be able to coordinate upstream/downstream stuff in whatever way we need. But I'm only one individual. <erikos> marcopg: that was the past <marcopg> mchua_: perhaps the best would be to try and talk with Kim <erikos> if there are no current issues - lets just forget about it <marcopg> mchua_: now we have a better plan, we didn't when we first discussed it <erikos> right <mchua_> And I certainly can't speak for all of OLPC, or even all of OLPC's QA team... I do currently liason between the OLPC test community and OLPC as part of my job responsibilities, so "officially" that's what I can speak as. <marcopg> I propose that we go ahead with plan. And if/when mchua_ has time she sync a bit OLPC management about it <erikos> sounds good to me <mchua_> erikos, marcopg, I think you're right; we don't have any current issues, we just resolved the *huge* one of Activity testing to the satisfaction of everyone here, we just need to propagate that "yay!" feeling out to SL and OLPC communities (and OLPC-HQ.) * cjb waves. <mchua_> marcopg proposal + 1 <erikos> i have learned - you have to act first <mchua_> ask forgiveness, not permissoin! <cjb> mchua_: that sounds right <marcopg> ok, fanstastic next point! <mchua_> s/permissoin/permission <marcopg> 4 mins left * mchua_ waves at cjb <erikos> if someone opposes he will do it - if not you were not blocked <mchua_> automation! <marcopg> hello cjb <erikos> hello cjb <marcopg> ok so my agenda for automation would be to figure out how we go forward <mchua_> I have no bandwidth for it! cjb proposed we send a call to the community on what automation we need! <marcopg> which team/people start looking into it etc <cjb> hi all. don't let me distract your last four minutes. :) <marcopg> the sugarbot guy seem to be back have time, btw <mchua_> on Sugar and Activity testing, I'd propose as a first step to have a sprint to try to spec out Things We Need To Build / Automate <erikos> yeah seen a blog post <mchua_> I don't think the problem space is well defined (from the perspective of OLPC and of SL, both) w00t sugarbot! <cjb> I think the lowest hanging fruit is sending out mail saying "hey this guy wrote this sugarbot thing and none of us have had time to look at it but I bet it'd make a good framework for activity testing" <marcopg> mchua_: I second that proposal <cjb> and, you know, you offer to make someone the head of the automated testing team, and so on <mchua_> as in, "I really don't know what we have/want/need re: automated Sugar/Activity testing for OLPC" - I have a reasonable idea, but it's likely to be out of date, and has huge holes. <marcopg> cjb: did someone actually look into sugarbot? <mchua_> cjb: +1, though I'd like to find ways to fit that into a larger, more coherent "so, Activity testing... here's the big picture" thing. <cjb> marcopg: I think we've all seen the screencast, but never tried running it <marcopg> I only have a vague idea of what it does <mchua_> marcopg: I did, a *veryveryvery* little bit. No patches. Just... I got it to run, once, on my desktop, which no longer has it installed (I wiped it and reinstalled ubuntu when it was getting flaky, months ago.) <marcopg> cjb: yeah was thinking more of code sanity :) <mchua_> marcopg: so, practically, "no." <mchua_> I also recall the setup process to be sort of painful for me, but that's probably due to n00bness in large part. I'd have to do it again to really say. <marcopg> so time is expired and my mum will get angry if I don't go to dinner :( pizza! <mchua_> (in other words, "I didn't document it, so it didn't happen.") <marcopg> what about if I take action to schedule an automation discussion in one of the next sugar development team meetings <cjb> mchua_: oh, you did more than us, at least :) <mchua_> marcopg: and I'll take the parallel action item to schedule the same for the olpc internal test group, with a warning that that will probably happen in January <marcopg> and pester mchua_, cjb, the sugarbot guy and other interested people to participate mchua_: sounds good ok great! we are done I think <mchua_> marcopg: agree <marcopg> thanks everyone for coming mchua_ in particular to make me much more clear about activities situation :) <mchua_> marcopg, your meeting-fu increases. :) <marcopg> happy to see things moving there! <mchua_> Likewise! <marcopg> mchua_: I will work, at some point! :) gah rock I need to learn to type, first _______________________________________________ IAEP -- It's An Education Project (not a laptop project!) [email protected] http://lists.sugarlabs.org/listinfo/iaep
