Hi Everyone, Thanks for the replies. Todd, I think that your 2 suggested updates, moving the traffic of gerrit to *[email protected] sounds good to me, and also the subscription email technique could also help more. Also my only other advice is periodically to send thoughts to [email protected] to give people a roadmap of things that matter to each of you and what you would want others outside of your circle to know about if they were to want to jump in. Sometimes as I mention below this is an email to the dev list (“Here’s what’s interesting to me, you may want to check out <these links> where <these links> could be Gerrit, JIRA, mailing list pointer, etc etc.”) This goes a long way towards welcoming us slow pokes to your minds and dev cycles and seeing where we can help.
I have also taken the time to reply individually to those emails that were sent. Somehow (mea culpa) I didn’t see the replies. Sigh. Cue laugh track. Replies to: Mike Percy Good comments about Hadoop and comparisons there. The thing is all projects are different in some sense, so comments to me like “Project X did it this way, so Y” may not mean that Kudu must do it that way. When I talk about design, I am not necessarily talking about design docs, but talking about the regular conversation and though process / roadmap indicating where things are going on. For me, that’s missing in Kudu - I’ve had several IM discussions with Todd about this, and he is able to give me a great roadmap of issue X is here at JIRA, and then cross referenced at Gerrit, and then it’s related to some code push in Git here, etc. etc. However as a casual reader of the dev list, my *main* entry point into ASF projects (and really as someone who’s been around the ASF since 2004, what I really consider the life-blood of the project), I would have and do not have a clue as to what is going on. I see daily heavy streams of Gerrit emails, and so forth on the dev list, but I do not see people talking to one another, saying things like: ==ExA: Hey KUDU-YYY is an important issue! I just merged it, check out the latest Git master and everyone give it a look-see. ==ExB: Hey Todd, you are in California and I’m in China - I have a design idea I wanted you to look at so I created this Y Gerrit review here, this JIRA issue Z here, and then this wiki page A here. Please have a look and give me feedback. Things like the above help people like me say, “oh, I want to be a part of that! Todd’s working on it; or Person B!” and then I’ll go have a look. Moreover, it also helps me help maintain shared community and ownership of the decisions and thoughts related to the code which should be shared by the community. It’s hard to follow when there is a small core of rapidly moving individuals who know what they want to do, and are all working at the same pace. I’m not paid to work on Kudu and so it’s hard for me to do that pace. Replies to Jean-Daniel Cryans: No it’s not the 90’s I just am not in that big open room you’re talking about. I’m in a room with bits and bytes whizzing by, linking to systems like Gerrit, and Slack convos, and other things which frankly I don’t have enough open tabs in my browser to keep track of and which don’t display nicely on my iPhone 6s mail app (which was built long after the 90s). In general though it seems like you got my point about getting better at using the dev list though and not just sending summaries there. Thanks. Replies to Adar Dembo: Thanks for your comments. Just realize that some day you may not be paid on a project to work on Kudu. Imagine then if you’d like to contribute to Kudu and how it could be more welcoming. I’m just trying to make that happen. OK team, thanks a lot for listening. Cheers, Chris ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chris Mattmann, Ph.D. Chief Architect Instrument Software and Science Data Systems Section (398) NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 168-519, Mailstop: 168-527 Email: [email protected] WWW: http://sunset.usc.edu/~mattmann/ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Director, Information Retrieval and Data Science Group (IRDS) Adjunct Associate Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA WWW: http://irds.usc.edu/ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -----Original Message----- From: jpluser <[email protected]> Reply-To: "[email protected]" <[email protected]> Date: Monday, March 14, 2016 at 5:28 PM To: "[email protected]" <[email protected]> Subject: Re: Creating more actual (non-code-auto-emails) discussion on the M/L >Phew, mea culpa! I have zero clue why I never received the replies >but I see them here: > >https://s.apache.org/vfw1 > >I’ll be reading the replies now. > >Thanks, Todd. > >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >Chris Mattmann, Ph.D. >Chief Architect >Instrument Software and Science Data Systems Section (398) >NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >Office: 168-519, Mailstop: 168-527 >Email: [email protected] >WWW: http://sunset.usc.edu/~mattmann/ >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >Director, Information Retrieval and Data Science Group (IRDS) >Adjunct Associate Professor, Computer Science Department >University of Southern California, Los Angeles, CA 90089 USA >WWW: http://irds.usc.edu/ >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > > > > >-----Original Message----- >From: Todd Lipcon <[email protected]> >Reply-To: "[email protected]" <[email protected]> >Date: Monday, March 14, 2016 at 5:22 PM >To: "[email protected]" <[email protected]> >Subject: Re: Creating more actual (non-code-auto-emails) discussion on the >M/L > >>Hey Chris, >> >>Maybe something's wrong with your email? There were several replies from >>JD >>Cryans, Mike Percy, Adar Dembo, and David Alves in this thread. >> >>-Todd >> >>On Mon, Mar 14, 2016 at 5:12 PM, Mattmann, Chris A (3980) < >>[email protected]> wrote: >> >>> I’m a bit worried that it’s been almost a week and not one person >>> has discussed this here besides Todd and/or I. There are 9 committers >>> and 7 mentors per [1]. Where is everybody? >>> >>> Cheers, >>> Chris >>> >>> [1] http://incubator.apache.org/projects/kudu.html >>> >>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> Chris Mattmann, Ph.D. >>> Chief Architect >>> Instrument Software and Science Data Systems Section (398) >>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >>> Office: 168-519, Mailstop: 168-527 >>> Email: [email protected] >>> WWW: http://sunset.usc.edu/~mattmann/ >>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> Director, Information Retrieval and Data Science Group (IRDS) >>> Adjunct Associate Professor, Computer Science Department >>> University of Southern California, Los Angeles, CA 90089 USA >>> WWW: http://irds.usc.edu/ >>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> >>> >>> >>> >>> >>> -----Original Message----- >>> From: jpluser <[email protected]> >>> Reply-To: "[email protected]" >>><[email protected]> >>> Date: Wednesday, March 9, 2016 at 8:00 PM >>> To: "[email protected]" <[email protected]> >>> Subject: Re: Creating more actual (non-code-auto-emails) discussion on >>>the >>> M/L >>> >>> >Thanks for the reply Todd. Unfortunately it's more systematic than >>>that. >>> >Apologies for top post but am on my phone. Couple points: >>> > >>> >- interested to hear from others besides you on this. No offense but I >>> >think it's important that project members send email here to reply. >>> > >>> >- I hand counted the 4 threads of interest. Didn't run a fancy command >>> >but to be honest it's more indicative of the broader issue. Things >>>aren't >>> >always solved through fancy greps and tools like gerrit. This is going >>>to >>> >be a core issue with Kudu's incubation - how is someone not sitting in >>>a >>> >cube working on the project who isn't on those tools like gerrit and >>> >slack which don't exist at the ASF going to join on the project? >>> > >>> >- Even considering 40 threads I doubt there have only be <= 40 >>> >*decisions* on the project to date. IOW they are being made somewhere >>>but >>> >it's unclear where. Email is easy to follow on a phone on the go >>> >whatever. >>> > >>> >As a mentor I would not be comfortable with Kudu being a TLP at this >>> >point bc frankly projects need to use their dev list for more than >>> >automated discussion and big reports. Simple as that and sending a >>> >transcript of where convo is happening elsewhere is not going to cut >>>it >>> >unfortunately. >>> > >>> >Email is slow and deliberate and not as fast or slick as gerrit etc, >>>but >>> >that's a good thing. It allows people the time needed to read and join >>>an >>> >OSS community. It's too hard to do that with Kudu right now. >>> > >>> >Cheers, >>> >Chris >>> > >>> >Sent from my iPhone >>> > >>> >> On Mar 9, 2016, at 6:46 PM, Todd Lipcon <[email protected]> wrote: >>> >> >>> >> Hey Chris, >>> >> >>> >> Responses inline: >>> >> >>> >> On Wed, Mar 9, 2016 at 6:08 PM, Mattmann, Chris A (3980) < >>> >> [email protected]> wrote: >>> >> >>> >>> Hi Team, >>> >>> >>> >>> I looked at: >>> >>> >>> >>> https://mail-archives.apache.org/mod_mbox/incubator-kudu-dev >>> >>> >>> >>> And over the last 4 months and Kudu’s inception, we have had >>> >>> well over 2k+ emails, and looking back I found 4 actual threads >>> >>> during that time (and one of which was a release VOTE thread) >>> >>> that wasn’t automatically generated by Gerrit. >>> >>> >>> >>> Mar 2016 438 >>> >>> Feb 2016 1003 >>> >>> Jan 2016 1143 >>> >>> Dec 2015 12 >>> >> >>> >> Hmm, I did a search in my inbox for: [email protected] >>> >>-gerrit >>> >> -jira -"git commit" -dev-help -"svn commit" -moderate -"git push >>> >>summary" >>> >> and counted 30-35 threads. You're right, of course, that JIRA and >>>gerrit >>> >> eclipse the amount of email discussion, though. >>> >> >>> >> >>> >>> >>> >>> >>> >>> If we are going to become an ASF top level project, the project >>> >>> discussion has to happen on the mailing list. We had similar >>> >>> issues in Spark and I realize that lots of project work is assisted >>> >>> by tools and other technologies, but at the ASF, “if it didn’t >>> >>> happen on the mailing list, it didn’t happen.” More-over it’s hard >>> >>> to parse signal from noise in all these automated messages. Frankly >>> >>> I don’t really know if anything good is going on - I know things >>> >>> are going on, and I assume they are good, but it’s extremely hard >>> >>> to verify that. >>> >> >>> >> I think it's worth noting that the "automated' messages are >>>typically >>> >>code >>> >> review requests and responses, which are developer discussion. Our >>> >> project's culture is usually to use JIRAs and/or 'work-in-progress' >>> >>patches >>> >> in gerrit to communicate when we find a bug or want an opinion on >>> >> something. For example, today I found a new bug >>> >> https://issues.apache.org/jira/browse/KUDU-1369 and wrote up a quick >>> >> work-in-progress for a a proposed solution and put it up at >>> >> http://gerrit.cloudera.org:8080/#/c/2514/ . I think it would be >>> >>redundant >>> >> to also send an email to the list saying "Hey guys, I found a bug, >>> >>here's a >>> >> description". >>> >> >>> >> The same goes for design discussion -- eg >>> >> http://gerrit.cloudera.org:8080/#/c/2443/ is a recent gerrit post >>>that >>> >>Dan >>> >> made for a new feature he's working on. In this case he also sent an >>> >>email >>> >> to the dev list to point out the gerrit in case anyone missed it. I >>> >>imagine >>> >> a lot of people would filter the gerrit emails out of their inbox >>>but >>> >>not >>> >> direct emails to the list (gerrit provides both headers and a >>>subject >>> >>line >>> >> tag to make it easy to do) >>> >> >>> >> In terms of daily dev discussion, most of it has been happening on >>>our >>> >> Slack -- eg earlier today three contributors were discussing >>>in-progress >>> >> efforts on Spark RDD integration and sharing code via that channel. >>> >>Most of >>> >> the community members we've seen so far have tended to prefer this >>>quick >>> >> back-and-forth for discussion. >>> >> >>> >> Of course any _decisions_ will be made on the mailing list. If you >>> >>think it >>> >> would be useful to send a daily slack log to the mailing list, we >>>can do >>> >> that as well. >>> >> >>> >> >>> >> >>> >>> >>> >>> I have a possible suggestions: >>> >>> >>> >>> * Create a [email protected] and send all automated >>> >>> traffic there. *-issues is one option; we could make another name >>>for >>> >>> it. >>> >> >>> >> Sure, we could do that. But, isn't it just as easy for people to set >>>up >>> >>a >>> >> filter for 'kudu-CR' if they want to move those messages elsewhere? >>>Our >>> >> initial motivation when setting up mailing lists was to avoid having >>>too >>> >> many (makes it a pain for people to subscribe to them all). >>> >> >>> >> >>> >>> >>> >>> That will help to separate the signal from the noise in terms of >>> >>> dev/architectural/etc. discussions from code reviews and automated >>> >>> commit messages. >>> >>> >>> >>> One thing you may say is that dev/architectural discussions are >>> >>>happening >>> >>> but they are in Gerrit. I would then say it’s extremely difficult >>>to >>> >>> separate the signal from the noise here, and as such, could be >>> >>>contributing >>> >>> towards making it difficult for others to join the project, >>>something >>> >>> that we identified as an issue in our Incubator report. >>> >> >>> >> Right. One option is that, for patches with bigger discussion, we >>>can >>> >>add a >>> >> gerrit "reviewer" which is actually the dev mailing list. This would >>> >>cause >>> >> the discussion to be CCed there, and bring it to the attention of >>>more >>> >> people. Another thought is to do as you suggest above and move >>>gerrit >>> >> elsewhere, and just have a policy that whenever any gerrit starts >>> >>getting >>> >> architectural, that we send a ping to the dev mailing list to point >>>it >>> >>out >>> >> (as Dan did with his recent design doc). >>> >> >>> >> -Todd >>> >>> >> >> >>-- >>Todd Lipcon >>Software Engineer, Cloudera >
