Re: Feedback on OCW on Windows
Hi Dominic, This is brilliant feedback. Can you open two separate JIRA tickets and we can work them through there? Also, see below for some comments, On Sun, Oct 7, 2018 at 7:58 PM wrote: > > First, it seems necessary to conda install proj4 in addition to ocw > (perhaps that should be a dependency?) If it needs to be installed then yes it certainly should be a dependency. Please submit a pull request and we can try it out. Maybe this has to do with Basemap packaging for Windows... I'm not sure. > > Second, it seems axes_grid1/axes_grid.py has this curious condition: > > if ngrids is None: > ngrids = self._nrows * self._ncols > else: > if not 0 <= ngrids < self._nrows * self._ncols: > raise Exception > > This exception happens (see below); that comparison looks off-by-one > (ngrids <= self._nrows * self._ncols). > I would need to step through the code to see the variable values, also this code is not for OCW so we would need to pick up the conversation over at MPL repository. You may have a also seen recently that Kyo opened an issue on JIRA recently on upgrading matplotlib as well https://issues.apache.org/jira/browse/CLIMATE-961 > > I'd love to help out, is there a pointer to how to develop with some local > packages, eg a local fork of axes_grid1, instead of conda versions? The pointer would be to clone that code and install it locally. Similarly you could install OCW from Git as well. > > All for now, > > Again thank you. If you could open JIRA issue for the above we can get started on this. Lewis
Re: Feedback on OCW on Windows
Hi Dominic, Responses inline: On Mon, Oct 8, 2018 at 1:31 AM Dominic Cooney wrote: > OK, I sorted things out to a degree. > > To summarize, here's problematic things for me as a Windows user: > > - When you clone from git and conda install ocw, and it leads to the > confusing situation I was in where the example is using the old conda ocw > package. > Maybe we should make this part more clear in the guide. The main purpose for cloning the repo is for having a quick way to download the test example scripts, NOT as a means for installing the library. See: https://cwiki.apache.org/confluence/display/CLIMATE/Installing+OCW+using+the+conda+package+manager > - The example seems to depend on "proj4" but that's not installed as an ocw > dependency. > proj4 is not listed explicitly listed as a dependency in our ocw conda recipe because we already list basemap which is, and proj4 is itself a dependency for basemap. conda should to be smart enough in these type of situations and automatically install proj4 for you if you don't have it. We can update our recipe to explicitly include proj4 if we must and we have done it in the past when conda was also not properly installing pyyaml from another dependency, but this is generally a bad practice and ideally should be reported and fixed upstream. > - matplotlib 2.2.2 mpl_toolkits/axes_grid1 has a bug where it won't produce > an n*m grid of plots. This appears to be fixed in later versions [1] > I can't reproduce this bug with my own matplotlib 2.2.2 installation. The condition for me shows: (ngrids > self._nrows * self._ncols) or (ngrids <= 0): When I crosscheck the versions on github, this is also what I am seeing. Are you certain that your version was 2.2.2? Also, any solution should not rely setting ngrid=None for all cases since it's necessary to ensure that the subplot layout doesn't draw empty axes rectangles if the number of plots to be drawn is less than size of the subplot matrix. Also, although most things are *supposed* to work correctly with Windows in theory, the reality is that many third party packages treat Windows as a second class citizen, in particular anything that requires compiling a C extension. For this reason I strongly recommend using the Windows Subsystem for Linux (WSL) if you are using Windows 10 rather than a native Windows based python installation. Anyways, please keep us posted with any other issues. We really appreciate it. Thanks, Alex > Here's some suggested improvements: > > -- conda install ocw --only-deps > -- conda install proj4 (* perhaps this should be an ocw dependency since > the example seems to depend on it?) > -- set PYTHONPATH to include the climate directory cloned from git > > I changed ocw/plotter.py:732 to pass ngrids=None to work around this > problem in mpl_toolkits/axes_grid1 and I now have some beautiful plots of > Africa. Updrading to matplotlib 3.0 unfortunately isn't a simple fix, > apparently matplotlib.cbook removed is_scalar which mpl_toolkits.basemap > depends on. I guess I'll follow up with matplotlib. > > [1] > > https://github.com/matplotlib/matplotlib/blob/98a0821b069ad75ad4b555ad15471dd6ffd14ae9/lib/mpl_toolkits/axes_grid1/axes_grid.py#L173 > > On Mon, Oct 8, 2018 at 11:58 AM Dominic Cooney > wrote: > > > Per the wiki's Getting Started page's [1] request, here's some trouble I > > ran into as a Windows user getting started. > > > > First, it seems necessary to conda install proj4 in addition to ocw > > (perhaps that should be a dependency?) otherwise mpl_toolkits.basemap > can't > > find epsg and this error results: > > > > C:\Users\coons\Documents\climate>python > > examples\simple_model_to_model_bias.py > > Traceback (most recent call last): > > File "examples\simple_model_to_model_bias.py", line 53, in > > import ocw.data_source.local as local > > File "C:\Program Files (x86)\Microsoft Visual > > Studio\Shared\Anaconda3_64\lib\site-packages\ocw\data_source\local.py", > > line 26, in > > from ocw.dataset import Dataset > > File "C:\Program Files (x86)\Microsoft Visual > > Studio\Shared\Anaconda3_64\lib\site-packages\ocw\dataset.py", line 34, in > > > > import ocw.utils as utils > > File "C:\Program Files (x86)\Microsoft Visual > > Studio\Shared\Anaconda3_64\lib\site-packages\ocw\utils.py", line 26, in > > > > from mpl_toolkits.basemap import shiftgrid, Basemap > > File "C:\Program Files (x86)\Microsoft Visual > > > Studio\Shared\Anaconda3_64\lib\site-packages\mpl_toolkits\basemap\__init__.py", > > line 146, in > > epsgf = open(os.path.join(pyproj.pyproj_datadir,'epsg')) > > FileNotFoundError: [Errno 2] No such file or directory: 'epsg' > > > > Second, it seems axes_grid1/axes_grid.py has this curious condition: > > > > if ngrids is None: > > ngrids = self._nrows * self._ncols > > else: > > if not 0 <= ngrids < self._nrows * self._ncols: > > raise Exception > > > > This exception happens (see below);
Re: Feedback on OCW on Windows
OK, I sorted things out to a degree. To summarize, here's problematic things for me as a Windows user: - When you clone from git and conda install ocw, and it leads to the confusing situation I was in where the example is using the old conda ocw package. - The example seems to depend on "proj4" but that's not installed as an ocw dependency. - matplotlib 2.2.2 mpl_toolkits/axes_grid1 has a bug where it won't produce an n*m grid of plots. This appears to be fixed in later versions [1]. Here's some suggested improvements: -- conda install ocw --only-deps -- conda install proj4 (* perhaps this should be an ocw dependency since the example seems to depend on it?) -- set PYTHONPATH to include the climate directory cloned from git I changed ocw/plotter.py:732 to pass ngrids=None to work around this problem in mpl_toolkits/axes_grid1 and I now have some beautiful plots of Africa. Updrading to matplotlib 3.0 unfortunately isn't a simple fix, apparently matplotlib.cbook removed is_scalar which mpl_toolkits.basemap depends on. I guess I'll follow up with matplotlib. [1] https://github.com/matplotlib/matplotlib/blob/98a0821b069ad75ad4b555ad15471dd6ffd14ae9/lib/mpl_toolkits/axes_grid1/axes_grid.py#L173 On Mon, Oct 8, 2018 at 11:58 AM Dominic Cooney wrote: > Per the wiki's Getting Started page's [1] request, here's some trouble I > ran into as a Windows user getting started. > > First, it seems necessary to conda install proj4 in addition to ocw > (perhaps that should be a dependency?) otherwise mpl_toolkits.basemap can't > find epsg and this error results: > > C:\Users\coons\Documents\climate>python > examples\simple_model_to_model_bias.py > Traceback (most recent call last): > File "examples\simple_model_to_model_bias.py", line 53, in > import ocw.data_source.local as local > File "C:\Program Files (x86)\Microsoft Visual > Studio\Shared\Anaconda3_64\lib\site-packages\ocw\data_source\local.py", > line 26, in > from ocw.dataset import Dataset > File "C:\Program Files (x86)\Microsoft Visual > Studio\Shared\Anaconda3_64\lib\site-packages\ocw\dataset.py", line 34, in > > import ocw.utils as utils > File "C:\Program Files (x86)\Microsoft Visual > Studio\Shared\Anaconda3_64\lib\site-packages\ocw\utils.py", line 26, in > > from mpl_toolkits.basemap import shiftgrid, Basemap > File "C:\Program Files (x86)\Microsoft Visual > Studio\Shared\Anaconda3_64\lib\site-packages\mpl_toolkits\basemap\__init__.py", > line 146, in > epsgf = open(os.path.join(pyproj.pyproj_datadir,'epsg')) > FileNotFoundError: [Errno 2] No such file or directory: 'epsg' > > Second, it seems axes_grid1/axes_grid.py has this curious condition: > > if ngrids is None: > ngrids = self._nrows * self._ncols > else: > if not 0 <= ngrids < self._nrows * self._ncols: > raise Exception > > This exception happens (see below); that comparison looks off-by-one > (ngrids <= self._nrows * self._ncols). > > I'd love to help out, is there a pointer to how to develop with some local > packages, eg a local fork of axes_grid1, instead of conda versions? Maybe > the sample just rotted a bit. Is there a unit test for the sample? > > Here's that exception stacktrace, FWIW: > > Generating a contour map using ocw.plotter.draw_contour_map() > Traceback (most recent call last): > File "simple_model_to_model_bias.py", line 164, in > subtitles=sub_titles) > File "C:\Program Files (x86)\Microsoft Visual > Studio\Shared\Anaconda3_64\lib\site-packages\ocw\plotter.py", line 734, in > draw_contour_map > cbar_pad='0%' > File "C:\Program Files (x86)\Microsoft Visual > Studio\Shared\Anaconda3_64\lib\site-packages\mpl_toolkits\axes_grid1\axes_grid.py", > line 489, in __init__ > raise Exception > Exception > > All for now, > > dpc > > [1] https://cwiki.apache.org/confluence/display/CLIMATE/Getting+Started >
