[jira] [Assigned] (CLIMATE-919) load_dataset_from_multiple_netcdf_files() does not have default variable units

[Alex Goodman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CLIMATE-919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Goodman reassigned CLIMATE-919:


Assignee: Alex Goodman

> load_dataset_from_multiple_netcdf_files() does not have default variable units
> --
>
> Key: CLIMATE-919
> URL: https://issues.apache.org/jira/browse/CLIMATE-919
> Project: Apache Open Climate Workbench
>  Issue Type: Bug
>  Components: data sources
>Affects Versions: 1.2.0
>Reporter: Alex Goodman
>Assignee: Alex Goodman
> Fix For: 1.3.0
>
>
> [~huikyole]
> load_dataset_from_multiple_netcdf_files() should read the variable units from 
> the netcdf file by default. Because it does not, when not entering them 
> manually in a config file I get this error:
> {code}
> TypeError Traceback (most recent call last)
> /home/goodman/rcmes/climate/RCMES/run_RCMES.py in ()
> 247 reference_dataset, reference_name, 
> target_datasets,
> 248 target_names,
> --> 249 
> path=workdir+config['output_netcdf_filename'])
> 250 
> 251 """ Step 7: Calculate metrics and draw plots """
> /home/goodman/rcmes/climate/ocw/dataset_processor.pyc in 
> write_netcdf_multiple_datasets_with_subregions(ref_dataset, ref_name, 
> model_dataset_array, model_names, path, subregions, subregion_array, 
> ref_subregion_mean, ref_subregion_std, model_subregion_mean, 
> model_subregion_std)
> 713 imodel].values
> 714 out_file.variables[model_names[imodel]].units = 
> model_dataset_array[
> --> 715 imodel].units
> 716 
> 717 if subregions is not None:
> netCDF4/_netCDF4.pyx in netCDF4._netCDF4.Variable.__setattr__ 
> (netCDF4/_netCDF4.c:38423)()
> netCDF4/_netCDF4.pyx in netCDF4._netCDF4.Variable.setncattr 
> (netCDF4/_netCDF4.c:35603)()
> netCDF4/_netCDF4.pyx in netCDF4._netCDF4._set_att (netCDF4/_netCDF4.c:8002)()
> TypeError: illegal data type for attribute, must be one of ['i8', 'f4', 'u8', 
> 'i1', 'u4', 'S1', 'i2', 'u1', 'i4', 'u2', 'f8'], got O
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (CLIMATE-919) load_dataset_from_multiple_netcdf_files() does not have default variable units

[Alex Goodman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CLIMATE-919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Goodman resolved CLIMATE-919.
--
Resolution: Fixed

> load_dataset_from_multiple_netcdf_files() does not have default variable units
> --
>
> Key: CLIMATE-919
> URL: https://issues.apache.org/jira/browse/CLIMATE-919
> Project: Apache Open Climate Workbench
>  Issue Type: Bug
>  Components: data sources
>Affects Versions: 1.2.0
>Reporter: Alex Goodman
> Fix For: 1.3.0
>
>
> [~huikyole]
> load_dataset_from_multiple_netcdf_files() should read the variable units from 
> the netcdf file by default. Because it does not, when not entering them 
> manually in a config file I get this error:
> {code}
> TypeError Traceback (most recent call last)
> /home/goodman/rcmes/climate/RCMES/run_RCMES.py in ()
> 247 reference_dataset, reference_name, 
> target_datasets,
> 248 target_names,
> --> 249 
> path=workdir+config['output_netcdf_filename'])
> 250 
> 251 """ Step 7: Calculate metrics and draw plots """
> /home/goodman/rcmes/climate/ocw/dataset_processor.pyc in 
> write_netcdf_multiple_datasets_with_subregions(ref_dataset, ref_name, 
> model_dataset_array, model_names, path, subregions, subregion_array, 
> ref_subregion_mean, ref_subregion_std, model_subregion_mean, 
> model_subregion_std)
> 713 imodel].values
> 714 out_file.variables[model_names[imodel]].units = 
> model_dataset_array[
> --> 715 imodel].units
> 716 
> 717 if subregions is not None:
> netCDF4/_netCDF4.pyx in netCDF4._netCDF4.Variable.__setattr__ 
> (netCDF4/_netCDF4.c:38423)()
> netCDF4/_netCDF4.pyx in netCDF4._netCDF4.Variable.setncattr 
> (netCDF4/_netCDF4.c:35603)()
> netCDF4/_netCDF4.pyx in netCDF4._netCDF4._set_att (netCDF4/_netCDF4.c:8002)()
> TypeError: illegal data type for attribute, must be one of ['i8', 'f4', 'u8', 
> 'i1', 'u4', 'S1', 'i2', 'u1', 'i4', 'u2', 'f8'], got O
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] climate pull request #452: CLIMATE 919 - load_dataset_from_multiple_netcdf_f...

[agoodm]

GitHub user agoodm opened a pull request:

https://github.com/apache/climate/pull/452

CLIMATE 919 - load_dataset_from_multiple_netcdf_files() does not have…

@huikyole 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/agoodm/climate CLIMATE-919

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/climate/pull/452.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #452


commit ba93542a75172b98f556747e520a0239ca8e6012
Author: Alex 
Date:   2017-09-26T20:38:30Z

CLIMATE 919 - load_dataset_from_multiple_netcdf_files() does not have 
default variable units




---

[GitHub] climate issue #452: CLIMATE 919 - load_dataset_from_multiple_netcdf_files() ...

[huikyole]

Github user huikyole commented on the issue:

https://github.com/apache/climate/pull/452
  
@agoodm I think this is good to go! Thx.


---

[GitHub] climate pull request #452: CLIMATE 919 - load_dataset_from_multiple_netcdf_f...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/climate/pull/452


---

[GitHub] climate pull request #453: CLIMATE-925 - CORDEX Config File Template

[agoodm]

GitHub user agoodm opened a pull request:

https://github.com/apache/climate/pull/453

CLIMATE-925 - CORDEX Config File Template

@huikyole The final script will come in just a bit (CLIMATE-926). However 
you can test these changes yourself by rendering the config file. Try running 
this script in the `climate/RCMES/CORDEX` directory. (You'll need jinja2, 
`conda install jinja2`):

```python
import jinja2
workdir = '/home/goodman/data_processing/CORDEX/analysis' # CHANGEME
from metadata_extractor import CORDEXMetadataExtractor, 
obs4MIPSMetadataExtractor
obs_extractor = obs4MIPSMetadataExtractor('/proj3/data/obs4mips')
models_extractor = CORDEXMetadataExtractor('/proj3/data/CORDEX/NAM-44/*')
groups = obs_extractor.group(models_extractor, 'variable')
obs_info, models_info = groups[0]
env =  jinja2.Environment(loader=jinja2.FileSystemLoader('./templates'), 
trim_blocks=True, lstrip_blocks=True)
t = env.get_template('CORDEX.yaml.template')

with open('test.yaml', 'w') as configfile:
configfile.write(t.render(obs_info=obs_info, models_info=models_info, 
season='annual', output_dir=workdir))
```

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/agoodm/climate CLIMATE-925

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/climate/pull/453.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #453


commit d868be10eb3862798f456e45e5ae2d3687dd401c
Author: Alex 
Date:   2017-09-26T23:03:09Z

Allow acronyms for CORDEX domains

commit 3e66d94fce1935aac2a8cc88025d50858d78f3ba
Author: Alex 
Date:   2017-09-26T23:10:44Z

Default vales for subplot grid in RCMES script

commit eacee1f5a2569a191d4961621d37281ccdf06017
Author: Alex 
Date:   2017-09-26T23:19:39Z

Allow acronyms for CORDEX domains

commit 8168d2042390712f9d58051633d93c267a002756
Author: Alex 
Date:   2017-09-26T23:21:09Z

Default values for filename_pattern in split file loader

commit 368e6969ca992e29b7bf49906449c4c92a913d2f
Author: Alex 
Date:   2017-09-26T23:33:14Z

Ignore start and end times when maximum_overlap is True

commit 3539aa2be6408140e33d82fb413089c62fcec70f
Author: Alex 
Date:   2017-09-26T23:41:06Z

CLIMATE-925 - CORDEX Config File Template




---