Is there anything specific I can help with regarding the new code base? Otherwise, I'll make a start on consolidating the examples and maybe move them to Jupyter.
On Sat, Nov 23, 2019 at 12:20 PM Lewis John McGibbney <lewi...@apache.org> wrote: > I also think that these are excellent tasks. > A cleanup of JIRA and branching legacy/unused code would be an excellent > start in order to cleanup master. > Lewis > > On 2019/11/21 11:09:53, Michael Anderson < > michael.arthur.ander...@gmail.com> wrote: > > To expand on that thought: > > > > Remove the UI. > > Remove anything related to virtualization and / or migrate it to AWS / > > Kubernetes. > > Remove any test or example that is not self contained (i.e. depends on > > inputs that are not publicly available). Shrink the examples to a > smaller > > number which exercise a larger number of features. > > Either move mccsearch to the examples or migrate the core functions to > the > > core library. > > > > Regarding the JIRA, I'd suggested that anything more than 2 years old can > > closed off. > > > > I'd suggest this would make it both easier for people looking to > contribute > > to find an impactful entry point and also simplify the migration path if > > it's to be consolidated into Alex's new library. > > > > On Thu, Nov 21, 2019 at 5:25 AM Michael Anderson < > > michael.arthur.ander...@gmail.com> wrote: > > > > > One of the challenges of working on the current project is that it > isn't > > > always clear which of the sub folders are deprecated / some of the > JIRA are > > > quite old and the original requestor is no longer interested. I'd > offer > > > that an aggressive pruning of the JIRA and sub folders would make it > easier > > > for people to engage on features that would be most impactful and long > > > lived. Barring that, that is what I find appealing about Alex's > > > suggestion as it clearly delineates what is no longer being supported > and > > > gives a clearer roadmap on where to make the most impactful > contributions. > > > > > > On Wed, Nov 20, 2019 at 4:49 PM Goodman, Alexander (US 398K) > > > <alexander.good...@jpl.nasa.gov.invalid> wrote: > > > > > >> Hi Lewis, > > >> > > >> I can't say I completely disagree with your assessment. I would go > even > > >> further and argue what you are saying has been true not just for 3 > months > > >> but for at least since the 1.3.0 release, if not longer. But to be > fair, > > >> most of the contributions to OCW including BCDP have been made > possible by > > >> JPL/NASA funding our labor. Even though Kyo and I have had some > success in > > >> obtaining a little bit more funding, it has been a very difficult > path when > > >> we are the only primary developers, compared to the distant past when > many > > >> more people were on board which is something I explain in the slides > I sent > > >> as being one of the primary motivators for developing BCDP. The hope > is > > >> that a more up to date API could alleviate some of the problems we > have > > >> been having with maintaining the codebase since it's much smaller > (xarray > > >> helps make many of the things we were previously doing more elegant > and > > >> concise after all...), but it is still not quite at the stage where > it can > > >> fully replace the current API, and I hope we can reach that final > step with > > >> just a bit more funding. > > >> > > >> As someone with more knowledge about ASF protocol than I, I do think > we > > >> should have a discussion about what our future software development > > >> practices should be, and we should definitely reach an agreement on > what > > >> the best approach for adding BCDP to our repository is (and CMDA for > that > > >> matter). I definitely don't want to just dump everything haphazardly > into > > >> separate subfolders like we always do (which is something I mentioned > in a > > >> recent JIRA issue thread that you may recall), but ideally we would > like to > > >> at least keep the OCW brand alive. However if things continue to > operate at > > >> the same sluggish pace and scale as they have been for sometime, then > > >> perhaps we should assess the merit of continuing to keep such a small > > >> project under the ASF's care. If we choose to declare the death of > OCW, so > > >> be it, but I will still release BCDP as a separate project outside > the ASF > > >> if I must. > > >> > > >> Kyo is on vacation for the rest of the week, but I am free to talk > with > > >> you more about this offline if you are around this week since I'd > like to > > >> have a very indepth discussion. Would you be interested? > > >> > > >> Thanks, > > >> Alex > > >> On 11/20/19, 12:25 PM, "Lewis John McGibbney" <lewi...@apache.org> > > >> wrote: > > >> > > >> This gets at my original statement. To expand however, let me > state, > > >> project activity is very low, new contributions are not being > reviewed, the > > >> roadmap is not clear, there seems to be a proposal to literally > drop-in > > >> replace current master OCW codebase with BCDP which has yet to be open > > >> sourced. > > >> None of this is particularly pro-community growth. > > >> I am not particularly happy with the way the community is being > > >> managed here at Apache. Nothing is particularly clear. The community > seems > > >> very stagnant and pretty much dead. This is in agreement with > > >> reporter.apache.org community health score which is -3.47 and > indicates > > >> that work is required. > > >> This thread actually is the most activity the project has seen in > the > > >> last 3 months! > > >> I had to file the PMC report this month... for those who want to > see > > >> what that looked like please monitor whimsy. > > >> I think some timelines backing up Alex's proposals are needed. > > >> Otherwise we are just delaying the slow death of OCW further. > > >> Lewis > > >> > > >> On 2019/11/20 16:52:47, Michael Anderson < > > >> michael.arthur.ander...@gmail.com> wrote: > > >> > That’s pretty interesting. I like adding incremental features > > >> with the goal of replacing rather than outright demise of the old > library. > > >> > > > >> > Sent from my iPhone > > >> > > > >> > > On Nov 20, 2019, at 11:08 AM, Goodman, Alexander (US 398K) > > >> <alexander.good...@jpl.nasa.gov.invalid> wrote: > > >> > > > > >> > > Since the plots didn't attach in the last message, here they > are: > > >> > > > > >> > > https://i.imgur.com/mAuq0R5.png > > >> > > https://i.imgur.com/TeRYSPI.png > > >> > > > > >> > > Thanks, > > >> > > Alex > > >> > > -----Original Message----- > > >> > > From: Goodman, Alexander (US 398K) > > >> <alexander.good...@jpl.nasa.gov.INVALID> > > >> > > Sent: Wednesday, November 20, 2019 8:05 AM > > >> > > To: dev@climate.apache.org > > >> > > Subject: RE: [EXTERNAL] Re: [DISCUSS] Retire OCW? > > >> > > > > >> > > Hi Michael, > > >> > > > > >> > > Please see this overview slide: > > >> > > https://i.imgur.com/VnijQ5C.png > > >> > > > > >> > > Here is also a link of some older slides which have a lot more > > >> details: > > >> > > > > >> > https://docs.google.com/presentation/d/1nhD3_fZmVcmBnq9NiMBYeDMsxr5kIimI7dE9NNUQJEM/edit?usp=sharing > > >> > > > > >> > > I also created an end-to-end processing benchmark that I > compared > > >> with the current OCW, which I have attached in two plots: One with > > >> regridding and one without, since the difference is so vast. I first > > >> presented these results at the AMS Annual meeting last year in > Phoenix. > > >> > > > > >> > > Let me know if you have any questions. > > >> > > > > >> > > Thanks, > > >> > > Alex > > >> > > -----Original Message----- > > >> > > From: Michael Anderson <michael.arthur.ander...@gmail.com> > > >> > > Sent: Tuesday, November 19, 2019 5:59 PM > > >> > > To: dev@climate.apache.org > > >> > > Subject: [EXTERNAL] Re: [DISCUSS] Retire OCW? > > >> > > > > >> > > Do you have an overview of the new library you could share? > It'd > > >> be interesting how big the gap between the old and new would be / > rough LOE > > >> to add parity. > > >> > > > > >> > >> On Tue, Nov 19, 2019 at 7:35 PM Alex Goodman < > good...@apache.org> > > >> wrote: > > >> > >> > > >> > >> Hi Lewis, > > >> > >> > > >> > >> To add some more specifics, the new xarray-based API I have > > >> developed > > >> > >> for OCW called BCDP (Big Climate Data Pipeline) has recently > > >> been > > >> > >> approved for release to open source. Keep in mind, this is a > > >> > >> completely new API that is not backwards compatible, so we > were > > >> > >> planning to maintain both the old and new API under one > > >> repository > > >> > >> (either as separate subfolders or git > > >> > >> branches) until the new API has enough functionality to fully > > >> replace > > >> > >> the old one. To accelerate this process and encourage more > > >> > >> development, both Kyo and I are seeking some additional > funding > > >> > >> sources which we should be more certain about early next > year. > > >> When I > > >> > >> first conceived BCDP, I was originally planning on releasing > it > > >> to > > >> > >> open source as a separate project outside of OCW, but after > some > > >> > >> discussion we thought that it would be better to maintain the > > >> OCW brand for it if possible. > > >> > >> > > >> > >> Does this make sense? > > >> > >> > > >> > >> Thanks, > > >> > >> Alex > > >> > >>> On 2019/11/20 00:17:22, lewis john mcgibbney < > > >> lewi...@apache.org> wrote: > > >> > >>> I got this from Kyo offline > > >> > >>> > > >> > >>> “There is enough motivation to keep OCW going. Alex has > almost > > >> > >>> refactored OCW and Seungwon Lee's CMDA will be a part of > OCW.” > > >> > >>> > > >> > >>> Some of us will remember that CMDA was retired from the > Apache > > >> > >>> Incubator previously. > > >> > >>> > > >> > >>> It would be great if these plans could be hashed out on the > > >> mailing > > >> > >>> list > > >> > >> so > > >> > >>> we can move forward either way. > > >> > >>> > > >> > >>> Lewis > > >> > >>> > > >> > >>> On Thu, Nov 14, 2019 at 09:32 Lewis John Mcgibbney < > > >> > >>> lewis.mcgibb...@gmail.com> wrote: > > >> > >>> > > >> > >>>> Hi Folks, > > >> > >>>> Project activity is very low and new contributions are not > > >> being > > >> > >> reviewed. > > >> > >>>> Simply, is it time to retire OCW to the Apache Attic? > > >> > >>>> Best > > >> > >>>> Lewis > > >> > >>>> > > >> > >>>> -- > > >> > >>>> > > >> > >>>> *Lewis* > > >> > >>>> Dr. Lewis J. McGibbney Ph.D, B.Sc > > >> > >>>> *Skype*: lewis.john.mcgibbney > > >> > >>>> > > >> > >>>> > > >> > >>>> > > >> > >>>> -- > > >> > >>> http://home.apache.org/~lewismc/ > > >> > >>> http://people.apache.org/keys/committer/lewismc > > >> > >>> > > >> > >> > > >> > > > >> > > >> > > >> > > >
