Re: PSA: JIRA resolutions and meanings

2016-10-08 Thread Holden Karau
We could certainly do that system - but given the current somewhat small set of active committers its clearly not scaling very well. There are many developers in Spark like Hyukjin, Cody, and myself who care about specific areas and can verify if an issue is still present in mainline. That being

Re: Improving volunteer management / JIRAs (split from Spark Improvement Proposals thread)

2016-10-08 Thread Cody Koeninger
Yeah, I've interacted with other projects that used that system and it was pleasant. 1. "this is getting closed cause its stale, let us know if thats a problem" 2. "actually that matters to us" 3. "ok well leave it open" I'd be fine with totally automating step 1 as long as a human was involved

Re: PSA: JIRA resolutions and meanings

2016-10-08 Thread Reynold Xin
On Sat, Oct 8, 2016 at 2:09 AM, Sean Owen wrote: > > - Resolve as Fixed if there's a change you can point to that resolved the > issue > - If the issue is a proper subset of another issue, mark it a Duplicate of > that issue (rather than the other way around) > - If it's

Re: PSA: JIRA resolutions and meanings

2016-10-08 Thread Ted Yu
Makes sense. I trust Hyukjin, Holden and Cody's judgement in respective areas. I just wish to see more participation from the committers. Thanks > On Oct 8, 2016, at 8:27 AM, Sean Owen wrote: > > Hyukjin

Re: PSA: JIRA resolutions and meanings

2016-10-08 Thread Felix Cheung
+1 on this proposal and everyone can contribute to updates and discussions on JIRAs Will be great if this could be put on the Spark wiki. On Sat, Oct 8, 2016 at 9:05 AM -0700, "Ted Yu" > wrote: Makes sense. I trust Hyukjin, Holden and

Re: PSA: JIRA resolutions and meanings

2016-10-08 Thread Sean Owen
I know what you mean Ted, and I think actually what's happening here is fine, but I'll explain my theory. There are a range of actions in the project where someone makes a decision, from answering an email to creating a release. We already accept that only some of these require a formal status;

Re: PSA: JIRA resolutions and meanings

2016-10-08 Thread Ted Yu
I think only committers should resolve JIRAs which were not created by himself / herself. > On Oct 8, 2016, at 6:53 AM, Hyukjin Kwon wrote: > > I am uncertain too. It'd be great if these are documented too. > > FWIW, in my case, I privately asked and told Sean first that

Re: PSA: JIRA resolutions and meanings

2016-10-08 Thread Cody Koeninger
So to be clear, can I go clean up the Kafka cruft? On Sat, Oct 8, 2016 at 1:41 PM, Reynold Xin wrote: > > On Sat, Oct 8, 2016 at 2:09 AM, Sean Owen wrote: >> >> >> - Resolve as Fixed if there's a change you can point to that resolved the >> issue >> - If

Re: PSA: JIRA resolutions and meanings

2016-10-08 Thread Reynold Xin
I think so (at least I think it is socially acceptable). Of course, use good judgement here :) On Sat, Oct 8, 2016 at 12:06 PM, Cody Koeninger wrote: > So to be clear, can I go clean up the Kafka cruft? > > On Sat, Oct 8, 2016 at 1:41 PM, Reynold Xin

Re: PSA: JIRA resolutions and meanings

2016-10-08 Thread Cody Koeninger
Cool, I'll start going through stuff as I have time. Already closed one, if anyone sees a problem let me know. Still think it would be nice to have some way to make it obvious to the people who have the will and knowledge to do it that it's ok for them to do it :) On Sat, Oct 8, 2016 at 2:19

Re: Improving governance / committers (split from Spark Improvement Proposals thread)

2016-10-08 Thread Cody Koeninger
It's not about technical design disagreement as to matters of taste, it's about familiarity with the domain. To make an analogy, it's as if a committer in MLlib was firmly intent on, I dunno, treating a collection of categorical variables as if it were an ordered range of continuous variables.

Re: Improving governance / committers (split from Spark Improvement Proposals thread)

2016-10-08 Thread Matei Zaharia
This makes a lot of sense; just to comment on a few things: > - More committers > Just looking at the ratio of committers to open tickets, or committers > to contributors, I don't think you have enough human power. > I realize this is a touchy issue. I don't have dog in this fight, > because I'm

Re: Spark Improvement Proposals

2016-10-08 Thread Matei Zaharia
Sounds good. Just to comment on the compatibility part: > I meant changing public user interfaces. I think the first design is > unlikely to be right, because it's done at a time when you have the > least information. As a user, I find it considerably more frustrating > to be unable to use a

Re: Improving volunteer management / JIRAs (split from Spark Improvement Proposals thread)

2016-10-08 Thread Matei Zaharia
I like this idea of asking them. BTW, one other thing we can do *provided the JIRAs are eventually under control* is to create a filter for old JIRAs that have not received a response in X amount of time and have the system automatically email the dev list with this report every month. Then

Re: Spark Improvement Proposals

2016-10-08 Thread vaquar khan
+1 for SIP lebles,waiting for Reynolds detailed proposal . Regards, Vaquar khan On 8 Oct 2016 16:22, "Matei Zaharia" wrote: > Sounds good. Just to comment on the compatibility part: > > > I meant changing public user interfaces. I think the first design is > >

PSA: JIRA resolutions and meanings

2016-10-08 Thread Sean Owen
That flood of emails means several people (Xiao, Holden mostly AFAICT) have been updating the status of old JIRAs. Thank you, I think that really does help. I have a suggested set of conventions I've been using, just to bring some order to the resolutions. It helps because JIRA functions as a

Re: PSA: JIRA resolutions and meanings

2016-10-08 Thread Cody Koeninger
That makes sense, thanks. One thing I've never been clear on is who should be allowed to resolve Jiras. Can I go clean up the backlog of Kafka Jiras that weren't created by me? If there's an informal policy here, can we update the wiki to reflect it? Maybe it's there already, but I didn't see

Re: PSA: JIRA resolutions and meanings

2016-10-08 Thread Hyukjin Kwon
I am uncertain too. It'd be great if these are documented too. FWIW, in my case, I privately asked and told Sean first that I am going to look though the JIRAs and resolve some via the suggested conventions from Sean. (Definitely all blames should be on me if I have done something terribly