Hi Michael, We actually discussed this in great detail today internally at JPL. I have not hosted the code anywhere yet, but am planning on putting it up on GitHub fairly soon once I have a more complete prototype and end-to-end example. When it's ready, I'll post a more formal announcement on the mailing list.
Thanks, Alex On Fri, Jan 26, 2018 at 5:45 PM, Michael Anderson < michael.arthur.ander...@gmail.com> wrote: > Alex, > > Have you posted the pandas version anywhere yet? I'd be happy to lend a > hand with the conversion if you have. > > Michael A. Anderson > > On Thu, Jan 18, 2018 at 10:17 AM, Goodman, Alexander (398K) < > alexander.good...@jpl.nasa.gov> wrote: > > > Hi Michael, > > > > This isn't something we have made any hard decisions about, but if you > > asked for my personal impression, I think just about everything outside > the > > core ocw library is more or less in a deprecated state. The original > > developers of most of the code in these subfolders have stopped > > contributing to the project long ago, and maintaining them has not been a > > high priority for the current developers. This sort of includes the UI > too > > since pretty much all of its features can be easily replicated (and much > > more easily maintained) inside a jupyter notebook or even a bokeh server > > (if you are familiar with those libraries) as it avoids the headaches of > > directly dealing with Node.js dependencies. This has actually been the > > subject of (mostly unsuccessful) proposals. Sort of like for my current > > experiment with pandas and xarray, anyone is of course free to work on > > something like this given that we are an open source project. It's just > > that for now, the rest of us have not had the time (or funding) to fully > > pursue it. > > > > So in regards to your question, the only thing external to the main ocw > > library which we still actively maintain is the RCMES subfolder, and the > > configuration file system it contains therein. But given that RCMES > itself > > is directly tied to JPL, this discussion has actually lead me to believe > > that it might be best for us to consider trimming down the main > repository > > to only include the main library and examples, while keeping everything > > RCMES-related (included issue-tracking ) as a separate set of > repositories. > > This is something that we should discuss some more in person during our > > planned dev meeting (@Kyo and Lewis). > > > > By the way, thank you very much for contributing to this discussion. > > Regrettably this dev mailing list in particular has been very quiet > lately > > but at least now we have had an opportunity to air out some important > > concerns regarding the future of the project. > > > > Thanks, > > Alex > > > > On Wed, Jan 17, 2018 at 2:49 AM, Michael Anderson < > > michael.arthur.ander...@gmail.com> wrote: > > > > > On a related topic, what's the roadmap for the features outside of main > > ocw > > > folder? Which of those will fold into the main ocw library, which will > > > remain as they are and which will be deprecated? > > > > > > > > > On Tue, Jan 16, 2018 at 11:33 PM, Goodman, Alexander (398K) < > > > alexander.good...@jpl.nasa.gov> wrote: > > > > > > > Hi Lewis, > > > > > > > > I think that is sort of a hasty action to take. If anything, I > question > > > > whether we should even require pydap as a dependency in the first > place > > > > since its main advantage (when used as a client-side tool, anyway) is > > to > > > > provide the ability to load OpenDAP datasets without the netcdf4 > > library > > > > (which is a core dependency to ocw and pretty much any other earth > > > science > > > > data analysis library out there). In the short term I think we could > > > modify > > > > the conda recipe to drop pydap for python 2.7, as that should fix our > > CI > > > > problems. > > > > > > > > However in the long term we will have no choice but to drop support > for > > > > python 2, as most of the pydata stack (ie, almost all of our core > > > > dependencies) will soon be doing so: > > > > > > > > http://www.python3statement.org/ > > > > > > > > This was actually something I was going to discuss with you and Kyo > in > > my > > > > planned meeting next week, as I have considered releasing the new > > version > > > > of ocw I am developing as a python 3-only codebase. In terms of our > > > current > > > > dependencies, only esgf-pyclient does not support python 3 (but this > > > should > > > > change soon enough). > > > > > > > > My one concern is that the scientific community in general has been > > > > resistant to switching (don't want to link anything here, since a > quick > > > > google search will yield numerous blog posts on the subject). Having > > > > personally been heavily involved in the process of making ocw python > > 3, I > > > > personally don't blame them since for those users python 3 just adds > > > > needless compatibility issues without offering them a whole lot of > > > > compelling new features. The most recent set of new features for > python > > > > 3.x, most notably asyncio (along with the new unicode string model, > > which > > > > is what actually breaks everything), are of far more use to web > > > developers > > > > than they are to scientists. If it weren't for the fact that the big > > > > projects I mentioned a few paragraphs ago are making a strong push > for > > > > python 3, then I am not sure I would even consider it for this > project > > > > given our type of userbase. The flipside to this is that as a core > > > > developer, I do understand that maintaining python2/3 support > > > > simultaneously is a rather messy process, and certainly not > something I > > > > would be too keen on doing when writing a new codebase in 2018. I'd > be > > > fine > > > > with maintaining a python 2.7-only codebase in an ideal world, but > > > > unfortunately that is not the world we are living in so for me > > > personally I > > > > believe there is only one other choice moving forward... > > > > > > > > So yeah, just my 2 cents. Ignoring my rant, here's the tl;dr: I am > > > against > > > > dropping python 2.7 support right away, but would be for gradually > > > phasing > > > > it out (especially if we decide to rewrite the codebase). > > > > > > > > PS: Which reminds me, the "new codebase" I keep mentioning is just a > > pet > > > > project of mine (currently dubbed "ocw2"), which makes gratuitous use > > of > > > > pandas and xarray as a data processing backend, rather than vanilla > > > > numpy/scipy. I'll discuss more details and hopefully open source my > > code > > > > within the next few weeks. > > > > > > > > Thanks, > > > > Alex > > > > > > > > On Tue, Jan 16, 2018 at 7:10 PM, lewis john mcgibbney < > > > lewi...@apache.org> > > > > wrote: > > > > > > > > > Hi Folks, > > > > > You will notice that our TravisCI builds currently report as being > > > > broken. > > > > > This is due to the pydap dependency not being available for Python > > 2.7. > > > > > Is it time to drop support for Python 2.7 altogether and simplify > > > things? > > > > > Lewis > > > > > > > > > > > > > > > -- > > > > > http://home.apache.org/~lewismc/ > > > > > http://people.apache.org/keys/committer/lewismc > > > > > > > > > > > > > > > > > > > > > -- > > > > Alex Goodman > > > > Data Scientist I > > > > Science Data Modeling and Computing (398K) > > > > Jet Propulsion Laboratory > > > > California Institute of Technology > > > > Tel: +1-818-354-6012 > > > > > > > > > > > > > > > -- > > Alex Goodman > > Data Scientist I > > Science Data Modeling and Computing (398K) > > Jet Propulsion Laboratory > > California Institute of Technology > > Tel: +1-818-354-6012 > > > -- Alex Goodman Data Scientist I Science Data Modeling and Computing (398K) Jet Propulsion Laboratory California Institute of Technology Tel: +1-818-354-6012
