Alex Goodman created CLIMATE-839:
------------------------------------
Summary: Further improvements to DatasetLoader
Key: CLIMATE-839
URL: https://issues.apache.org/jira/browse/CLIMATE-839
Project: Apache Open Climate Workbench
Issue Type: Improvement
Components: config, data sources
Affects Versions: 1.1.0
Reporter: Alex Goodman
Assignee: Alex Goodman
Fix For: 1.2.0
I sat down with [~huikyole] and we agreed on the following improvements to
{{DatasetLoader}}:
1) Stop distinguishing between reference and target datasets. This was only
necessary for certain metrics, but in our mission to make the process of
dataset loading more flexible we thought it would be better to keep things as
general as possible at this step. Furthermore, we would also like to make it
possible to process multiple observation datasets at once.
2) Set default loader to local. By default, if a data_source isn't specified, a
KeyError is thrown. This will be changed to using the {{load_multiple_files}}
loader from local. Furthermore, we do not need to specify 'local_multiple' and
'local' separately since the latter loader can easily handle the single file
case, so this will be the new default loader rather than {{local.load_file}}.
3) Change 'data_source' keyword to 'loader_name'. This needs to be done at the
config file level too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
