[
https://issues.apache.org/jira/browse/CLIMATE-838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15412870#comment-15412870
]
ASF GitHub Bot commented on CLIMATE-838:
----------------------------------------
GitHub user agoodm opened a pull request:
https://github.com/apache/climate/pull/384
CLIMATE-838 - Update configuration files and example scripts to use new
DatasetLoader
This PR makes the following changes:
- Datasets are now loaded from the new `DatasetLoader` class in
`run_RCMES.py`.
- I also cleaned up `run_RCMES.py` to make it more closely conform to PEP8,
in particular I made many blocks of code multiline to make them more readable.
There is still much more that can be done, but this seems like a good start.
- Several variable names had to be changed in the config files in order to
make them compatible with `DatasetLoader`. Also note that new blocks of info
for each dataset loader are now denoted by a dash (-) under `reference` and
`targets`. This will make it easy to use multiple loaders in each file.
Note that the capability of handling more than one reference (observation)
dataset is still not included, I am saving that for a separate JIRA issue / PR.
Right now getting these fundamental changes out of the way and making sure
everything else works is a bigger priority.
So far I have only tested these changes for one config file
(`RCMES/configuration_files/cordex_AF_prec_subregion_annual_cycle_time_series.yaml`).
@huikyole, please feel free to make comments and keep me posted here.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/agoodm/climate CLIMATE-838
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/climate/pull/384.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #384
----
commit 711f86ba3533aa42cfd8190aac8edcc1e814413f
Author: Alex Goodman <ago...@users.noreply.github.com>
Date: 2016-08-03T17:20:26Z
Changed all instances of 'data_source' to 'loader_name'
commit 9797d97152a2cc8bcd46a0d05ff5c56645026bda
Author: Alex Goodman <ago...@users.noreply.github.com>
Date: 2016-08-04T23:28:50Z
Resolve merge conflict with CLIMATE-827
commit 8553dfe341d6abe1e101729d32fdc710014d254b
Author: Alex Goodman <ago...@users.noreply.github.com>
Date: 2016-08-08T23:53:07Z
Update run_RCMES.py for new DatasetLoader w/ minor config file API changes
commit 93971e16fcfd9823603a436fe6c7f24cc8ee72e1
Author: Alex Goodman <ago...@users.noreply.github.com>
Date: 2016-08-09T00:58:42Z
Fixed bugs in run_RCMES.py via testing config files
----
> Update configuration files and example scripts to use new DatasetLoader
> -----------------------------------------------------------------------
>
> Key: CLIMATE-838
> URL: https://issues.apache.org/jira/browse/CLIMATE-838
> Project: Apache Open Climate Workbench
> Issue Type: Improvement
> Components: config, data sources
> Affects Versions: 1.1.0
> Reporter: Alex Goodman
> Assignee: Huikyo Lee
> Fix For: 1.2.0
>
>
> The new {{DatasetLoader}} class found in {{dataset_loader.py}} was designed
> to do the following:
> 1) Unify the loader functions for each data source in one module.
> 2) Greatly simplify the process of parsing input from configuration files
> into the loaders.
> 3) Allow users to easily load datasets from many different data sources in a
> single evaluation run.
> The unit tests I have written can test some of this functionality, but it is
> now time to put it into practice. This means updating the configuration files
> and the associated evaluation scripts (eg run_RCMES.py).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
