Re: A renewed plea for help [was Re: Recruiting more maintainers for Apache Arrow]

2019-01-24 Thread Wes McKinney
hi Antoine, On Wed, Jan 23, 2019 at 4:35 AM Antoine Pitrou wrote: > > On Tue, 22 Jan 2019 16:57:42 -0600 > Wes McKinney wrote: > > > > There were 1540 patches merged into the project in 2018 (excluding the > > Parquet merge) -- that's more than 4 patches per day. Evidence > > suggests that the

Re: A renewed plea for help [was Re: Recruiting more maintainers for Apache Arrow]

2019-01-23 Thread Antoine Pitrou
On Tue, 22 Jan 2019 16:57:42 -0600 Wes McKinney wrote: > > There were 1540 patches merged into the project in 2018 (excluding the > Parquet merge) -- that's more than 4 patches per day. Evidence > suggests that the overall patch count for 2019 will be even higher; if > I had to guess somewhere

Re: A renewed plea for help [was Re: Recruiting more maintainers for Apache Arrow]

2019-01-22 Thread Micah Kornfield
Hi Wes, I can take a stab at going through the older C++/Python PRs as a first pass triage (I also appreciate that I'm just getting back to project, so if you prefer I hold off on this I understand). Is there a good mechanism to have a committer/PMC member look at PRs after a first pass by a

Re: A renewed plea for help [was Re: Recruiting more maintainers for Apache Arrow]

2019-01-22 Thread Wes McKinney
hi folks, It's been 3 months since I sent this e-mail so I thought I would follow up about where things stand. The project continues to grow very fast and so there is a ton of pull request and JIRA gardening to do. For the 0.12.0 release, my personal burden of patch-merging stood at about 50%.

A renewed plea for help [was Re: Recruiting more maintainers for Apache Arrow]

2018-10-15 Thread Wes McKinney
hi folks, It's been a few months, but as Apache Arrow is rapidly becoming a critical dependency of next-generation data applications (see, for example, RAPIDS just launched by NVIDIA http://rapids.ai/), we are quite seriously in need of more project maintainers, or in lieu of new individual

Re: Recruiting more maintainers for Apache Arrow

2018-07-02 Thread Antoine Pitrou
Hi, Le 02/07/2018 à 15:58, Wes McKinney a écrit : > * http://ivory.idyll.org/blog/2018-how-open-is-too-open.html > * http://ivory.idyll.org/blog/2018-oss-framework-cpr.html Very good articles, but I would stress that some of the mechanisms proposed lack metrics in their favour. Two particular

Re: Recruiting more maintainers for Apache Arrow

2018-07-02 Thread Wes McKinney
Hi folks, I would like to highlight that the challenges we are having are endemic to many parts of the open source world right now. A colleague of mine in the Python world wrote some pieces about this recently: * http://ivory.idyll.org/blog/2018-how-open-is-too-open.html *

Re: Recruiting more maintainers for Apache Arrow

2018-07-02 Thread Antoine Pitrou
Hi Dimitri, Le 02/07/2018 à 12:46, Dimitri Vorona a écrit : > Hi Wes, > > to contribute an outsiders POW: while it is clear, what's expected if you'd > like to make a PR, it's not at all clear to me, where would I start if I > wanted to help with PR reviews without being heavily involved with

Re: Recruiting more maintainers for Apache Arrow

2018-07-02 Thread Dimitri Vorona
Hi Wes, to contribute an outsiders POW: while it is clear, what's expected if you'd like to make a PR, it's not at all clear to me, where would I start if I wanted to help with PR reviews without being heavily involved with the community/being a full maintainer. Should I just grab a PR, test it,

Re: Recruiting more maintainers for Apache Arrow

2018-06-30 Thread Donald E. Foss
For what it's worth, this email thread and your summary writeup, Wes, are a significant call to action on their own. I've been passive, not by choice, but by policy. Given the significance and need of this project, I'll see what I can do on my side. It will be at least a week given the US

Re: Recruiting more maintainers for Apache Arrow

2018-06-30 Thread Wes McKinney
hi Antoine, On Sat, Jun 30, 2018 at 2:35 PM, Antoine Pitrou wrote: > > Hi Wes, > >> I'm not sure what's the best way to address this problem. The quality >> of our code review has declined at times as we struggle to keep up >> with the flow of patches -- I don't think this is good. Having the >>

Re: Recruiting more maintainers for Apache Arrow

2018-06-30 Thread Wes McKinney
hi Marco, some comments inline On Sat, Jun 30, 2018 at 2:15 PM, Marco Neumann wrote: > Hey, > > first of all, thanks a lot for your, Uwes, the mergers and contributors > work. Now, to the maintainer problem: > > # Arrow as "a library" > One thing that makes Arrow special is that it is not a

Re: Recruiting more maintainers for Apache Arrow

2018-06-30 Thread Marco Neumann
Hey, first of all, thanks a lot for your, Uwes, the mergers and contributors work. Now, to the maintainer problem: # Arrow as "a library" One thing that makes Arrow special is that it is not a single, but many libraries (one for each language) and many of them are not only a binding to a C/C++

Re: Recruiting more maintainers for Apache Arrow

2018-06-30 Thread Antoine Pitrou
Hi Wes, > I'm not sure what's the best way to address this problem. The quality > of our code review has declined at times as we struggle to keep up > with the flow of patches -- I don't think this is good. Having the > patch queue pile up isn't great either. I'd like to do more reviews but

Re: Recruiting more maintainers for Apache Arrow

2018-06-30 Thread Marco Neumann
Hey, first of all, thanks a lot for your, Uwes, the mergers and contributors work. Now, to the maintainer problem: # Arrow as "a library" One thing that makes Arrow special is that it is not a single, but many libraries (one for each language) and many of them are not only a binding to a C/C++

Re: Recruiting more maintainers for Apache Arrow

2018-06-30 Thread Holden Karau
One of the things I’ve started doing in the Spark project is live code reviews to encourage other folks to get involved in the review process and help it seem more achievable (see https://www.youtube.com/playlist?list=PLRLebp9QyZtYF46jlSnIu2x1NDBkKa2uw ) . Another that I think has helped us is

Recruiting more maintainers for Apache Arrow

2018-06-30 Thread Wes McKinney
hi folks, Arrow has grown by leaps and bounds over the last 2.5 years. We are approaching our 2000th patch and on track to surpass 200 unique contributors by year end. All this contribution growth is great, but it has a hidden cost: the maintenance. The burden of maintaining the project: