Hi John, We have now constructed a real netCDF file for you to check. QC flag variables have been added in. The QC flagging method is based on the proposed IODE scheme in (1) below.
Attached are the sample .nc file, text (ncdump) of same and a document specifying the CDL. Thanks and Regards, Andrew Walsh Ref.(1) Konovalov et. al (March 2012), Proposal to adopt a quality flag scheme standard
for oceanographic and marine meteorological data, Version 1.2.----- Original Message ----- From: "John Caron" <[email protected]>
To: "andrew walsh" <[email protected]> Sent: Thursday, April 05, 2012 10:44 AM Subject: Re: [CF-metadata] Ocean CTD data following CF Conventions v1.6
ok On 4/4/2012 6:42 PM, andrew walsh wrote:Hi John,Thank for your offer to check a sample netCDF CTD data file. At moment we don't havesome real .nc file but when we do I can send you a file for checking. Andrew ----- Original Message ----- From: John Caron To: [email protected] Sent: Wednesday, April 04, 2012 5:55 AM Subject: Re: [CF-metadata] Ocean CTD data following CF Conventions v1.6 Hi all: Lets see, I havent followed the entire conversation, but:1) Andrew if you can send me a sample file (not just the CDL) I can check if it works in the CDM with the new 1.6 conventions, and maybe give you some advice from my POV.2) Aggregation in the CDM comes in 2 flavors. 1) The original implementation simply appends multidimensional arrays together, eg "joinNew " and "joinExisting" NCML aggregation. I call it "syntactic aggregation" because it doesnt know what its aggregating, and the homogeneity requirements are strict. 2) "Feature Type collections" (aka "semantic aggregation") are the more recent development. These understand the coordinate information of the data, and so can handle variations in the representation, eg ragged arrays, scalar coordinates, etc. In this case, the CDM understands a dataset as a collection of "features objects", which can be stored in a collection of files. The interfaces to these collections is still under development. Most current and future work in the CDM is in this category.John snip .... _______________________________________________ CF-metadata mailing list [email protected] http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
netcdf 20110818T001140Z_M_HI504BEN {
dimensions:
time = 1 ;
pressure = UNLIMITED ; // (9 currently)
latitude = 1 ;
longitude = 1 ;
variables:
double time(time) ;
time:standard_name = "time" ;
time:units = "days since 1950-01-01 00:00:00Z" ;
time:axis = "T" ;
time:valid_min = 0 ;
time:valid_max = 999999 ;
byte time_qc_flag ;
time_qc_flag:long_name = "quality control flag for time (primary
Level 1 flag)" ;
time_qc_flag:quality_control_convention = "Proposed IODE qc scheme
March 2012" ;
time_qc_flag:valid_min = 1 ;
time_qc_flag:valid_max = 9 ;
time_qc_flag:flag_values = 1b, 2b, 3b, 4b, 9b ;
time_qc_flag:flag_meanings = "good not_evaluated_or_unknown suspect
bad missing" ;
double latitude(latitude) ;
latitude:standard_name = "latitude" ;
latitude:units = "degrees_north" ;
latitude:axis = "Y" ;
latitude:valid_min = -90 ;
latitude:valid_max = 90 ;
double longitude(longitude) ;
longitude:standard_name = "longitude" ;
longitude:units = "degrees_east" ;
longitude:axis = "X" ;
longitude:valid_min = -180 ;
longitude:valid_max = 180 ;
byte position_qc_flag ;
position_qc_flag:long_name = "quality control flag for position
(primary Level 1 flag)" ;
position_qc_flag:quality_control_convention = "Proposed IODE qc
scheme March 2012" ;
position_qc_flag:valid_min = 1 ;
position_qc_flag:valid_max = 9 ;
position_qc_flag:flag_values = 1b, 2b, 3b, 4b, 9b ;
position_qc_flag:flag_meanings = "good not_evaluated_or_unknown
suspect bad missing" ;
double pressure(pressure) ;
pressure:standard_name = "sea_water_pressure" ;
pressure:units = "decibars" ;
pressure:axis = "Z" ;
pressure:valid_min = 0 ;
pressure:valid_max = 12000 ;
pressure:positive = "down" ;
double temperature(pressure) ;
temperature:_FillValue = -99.99 ;
temperature:standard_name = "sea_water_temperature" ;
temperature:units = "degrees_C" ;
temperature:valid_min = -2 ;
temperature:valid_max = 40 ;
temperature:ancillary_variables = "temperature_whole_profile_flag
temperature_qc_flag temperature_sd_test" ;
temperature:coordinates = "time latitude longitude pressure" ;
byte temperature_whole_profile_flag ;
temperature_whole_profile_flag:long_name = "qc flag for whole
temperature profile (primary Level 1 flag)" ;
temperature_whole_profile_flag:quality_control_convention =
"Proposed IODE qc scheme March 2012" ;
temperature_whole_profile_flag:valid_min = 1 ;
temperature_whole_profile_flag:valid_max = 9 ;
temperature_whole_profile_flag:flag_values = 1b, 2b, 3b, 4b, 9b
;
temperature_whole_profile_flag:flag_meanings = "good
not_evaluated_or_unknown suspect bad missing" ;
byte temperature_qc_flag(pressure) ;
temperature_qc_flag:long_name = "quality control flag for
temperature (primary Level 1 flag)" ;
temperature_qc_flag:standard_name = "sea_water_temperature
status_flag" ;
temperature_qc_flag:quality_control_convention = "Proposed IODE qc
scheme March 2012" ;
temperature_qc_flag:valid_min = 1 ;
temperature_qc_flag:valid_max = 9 ;
temperature_qc_flag:flag_values = 1b, 2b, 3b, 4b, 9b ;
temperature_qc_flag:flag_meanings = "good not_evaluated_or_unknown
suspect bad missing" ;
temperature_qc_flag:coordinates = "time latitude longitude
pressure" ;
byte temperature_sd_test(pressure) ;
temperature_sd_test:long_name = "qc flag for monthly temperature 3
standard deviation test (secondary Level 2 flag)" ;
temperature_sd_test:quality_control_convention = "Proposed IODE qc
scheme March 2012" ;
temperature_sd_test:valid_min = 0 ;
temperature_sd_test:valid_max = 2 ;
temperature_sd_test:flag_values = 0b, 1b, 2b ;
temperature_sd_test:flag_meanings = "passed failed unknown" ;
temperature_sd_test:coordinates = "time latitude longitude
pressure" ;
double salinity(pressure) ;
salinity:_FillValue = -99.99 ;
salinity:standard_name = "sea_water_salinity" ;
salinity:units = "psu" ;
salinity:valid_min = 0 ;
salinity:valid_max = 45 ;
salinity:ancillary_variables = "salinity_whole_profile_flag
salinity_qc_flag salinity_sd_test" ;
salinity:coordinates = "time latitude longitude pressure" ;
byte salinity_whole_profile_flag ;
salinity_whole_profile_flag:long_name = "qc flag for whole salinity
profile (primary Level 1 flag)" ;
salinity_whole_profile_flag:quality_control_convention = "Proposed
IODE qc scheme March 2012" ;
salinity_whole_profile_flag:valid_min = 1 ;
salinity_whole_profile_flag:valid_max = 9 ;
salinity_whole_profile_flag:flag_values = 1b, 2b, 3b, 4b, 9b ;
salinity_whole_profile_flag:flag_meanings = "good
not_evaluated_or_unknown suspect bad missing" ;
byte salinity_qc_flag(pressure) ;
salinity_qc_flag:long_name = "quality control flag for salinity
(primary Level 1 flag)" ;
salinity_qc_flag:standard_name = "sea_water_salinity
status_flag" ;
salinity_qc_flag:quality_control_convention = "Proposed IODE qc
scheme March 2012" ;
salinity_qc_flag:valid_min = 1 ;
salinity_qc_flag:valid_max = 9 ;
salinity_qc_flag:flag_values = 1b, 2b, 3b, 4b, 9b ;
salinity_qc_flag:flag_meanings = "good not_evaluated_or_unknown
suspect bad missing" ;
salinity_qc_flag:coordinates = "time latitude longitude
pressure" ;
byte salinity_sd_test(pressure) ;
salinity_sd_test:long_name = "qc flag for monthly salinity 3
standard deviation test (secondary Level 2 flag)" ;
salinity_sd_test:quality_control_convention = "Proposed IODE qc
scheme March 2012" ;
salinity_sd_test:valid_min = 0 ;
salinity_sd_test:valid_max = 2 ;
salinity_sd_test:flag_values = 0b, 1b, 2b ;
salinity_sd_test:flag_meanings = "passed failed unknown" ;
salinity_sd_test:coordinates = "time latitude longitude
pressure" ;
int profile ;
profile:long_name = "profile identifier" ;
profile:cf_role = "profile_id" ;
// global attributes:
:conventions = "CF-1.6" ;
:featureType = "profile" ;
:cdm_data_type = "Profile" ;
data:
time = 22509.0081018517 ;
time_qc_flag = 1 ;
latitude = -10.5688 ;
longitude = 142.11713 ;
position_qc_flag = 1 ;
pressure = 4, 5, 6, 7, 8, 9, 10, 11, 12 ;
temperature = 24.737, 24.737, 24.739, 24.738, 24.737, 24.736, 24.7375,
24.737, 24.735 ;
temperature_whole_profile_flag = 1 ; temperature_qc_flag = 1, 1, 1, 1, 1, 1, 1, 1, 1 ; temperature_sd_test = 0, 0, 0, 0, 0, 0, 0, 0, 0 ;salinity = 35.2028, 35.2032, 35.2013, 35.2018, 35.2022, 35.2019, 35.2019, 35.2024, 35.204 ;
salinity_whole_profile_flag = 1 ; salinity_qc_flag = 1, 1, 1, 1, 1, 1, 1, 1, 1 ; salinity_sd_test = 0, 0, 0, 0, 0, 0, 0, 0, 0 ; profile = 123456 ; }
20110818T001140Z_M_HI504BEN.nc
Description: Binary data
****CDL representation of QC'ed CTD ocean profile data at RAN METOC Data Centre***** The following is a proposed CDL for a netCDF file describinga quality controlled vertical profile of pressure, temperature and salinity collected by a CTD instrument. The CDL follows the netCDF CF coventions version 1.6 at ref (2). The CTD data is classed
as featureType 'profile' as per Chapter 9 'Discrete sampling geometries' in ref (2). The quality control (QC) flagging scheme follows the 2 level QC flag scheme proposed by IODE in the ref (1) below. In this QC scheme the Level 1 flag is the primary QC flag. Level 2 flags are secondary indicating results of specified QC tests. Level 2 flags use a numerical code i.e 0 (passed),1 (failed) or 2(unknown) to indicate test status of a specified test.
Refs (1) Konovalov et. al (March 2012), Proposal to adopt a quality flag scheme standard for oceanographic and marine meteorological data, Version 1.2. (2) http://cf-pcmdi.llnl.gov/documents/cf-conventions dimensions: time = 1 ; pressure = UNLIMITED ; // (9 currently) latitude = 1 ; longitude = 1 ; variables: double time(time) ; time:standard_name = "time" ; time:units = "days since 1950-01-01 00:00:00Z" ; time:axis = "T" ; time:valid_min = 0 ; time:valid_max = 999999 ; byte time_qc_flag; time_qc_flag:long_name = "quality control flag for time (Level 1 flag)" ; time_qc_flag:quality_control_convention = "Proposed IODE qc scheme March 2012" ; time_qc_flag:valid_min = 1 ; time_qc_flag:valid_max = 9 ; time_qc_flag:flag_values = 1b, 2b, 3b, 4b, 9b ; time_qc_flag:flag_meanings = "good not_evaluated_or_unknown suspect bad missing" ; double latitude(latitude) ; latitude:standard_name = "latitude" ; latitude:units = "degrees_north" ; latitude:axis = "Y" ; latitude:valid_min = -90 ; latitude:valid_max = 90 ; double longitude(longitude) ; longitude:standard_name = "longitude" ; longitude:units = "degrees_east" ; longitude:axis = "X" ; longitude:valid_min = -180 ; longitude:valid_max = 180 ; byte position_qc_flag; position_qc_flag:long_name = "quality control flag for position (Level 1 flag)" ; position_qc_flag:quality_control_convention = "Proposed IODE qc scheme March 2012" ; position_qc_flag:valid_min = 1 ; position_qc_flag:valid_max = 9 ; position_qc_flag:flag_values = 1b, 2b, 3b, 4b, 9b ; position_qc_flag:flag_meanings = "good not_evaluated_or_unknown suspect bad missing" ; double pressure(pressure) ; pressure:standard_name = "sea_water_pressure" ; pressure:units = "decibars" ; pressure:axis = "Z" ; pressure:valid_min = 0 ; pressure:valid_max = 12000 ; pressure:positive = "down" ; double temperature(pressure) ; temperature:_FillValue = -99.99 ; temperature:standard_name = "sea_water_temperature" ; temperature:units = "degrees_C" ; temperature:valid_min = -2 ; temperature:valid_max = 40 ; temperature:ancillary_variables = "temperature_whole_profile_flag temperature_qc_flag temperature_sd_test" ; temperature:coordinates = "time latitude longitude pressure" ; byte temperature_whole_profile_flag ; temperature_whole_profile_flag:long_name = "qc flag for whole temperature profile (primary L1 flag)" ; temperature_whole_profile_flag:quality_control_convention = "IODE qc scheme March 2012" ; temperature_whole_profile_flag:valid_min = 1 ; temperature_whole_profile_flag:valid_max = 9 ; temperature_whole_profile_flag:flag_values = 1b, 2b, 3b, 4b, 9b ; temperature_whole_profile_flag:flag_meanings = "good not_evaluated_or_unknown suspect bad missing" ; byte temperature_qc_flag(pressure) ; temperature_qc_flag:long_name = "quality control flag for temperature (primary Level 1 flag)" ; temperature_qc_flag:standard_name = "sea_water_temperature status_flag" ; temperature_qc_flag:quality_control_convention = "IODE qc scheme March 2012" ; temperature_qc_flag:valid_min = 1 ; temperature_qc_flag:valid_max = 9 ; temperature_qc_flag:flag_values = 1b, 2b, 3b, 4b, 9b ; temperature_qc_flag:flag_meanings = "good not_evaluated_or_unknown suspect bad missing" ; temperature_qc_flag:coordinates = "time latitude longitude pressure" ; byte temperature_sd_test(pressure) ; temperature_sd_test:long_name = "qc flag for monthly temperature 3 standard deviation test (secondary L2 flag)" temperature_sd_test:quality_control_convention = "IODE qc scheme March 2012" ; temperature_sd_test:valid_min = 0 ; temperature_sd_test:valid_max = 2 ; temperature_sd_test:flag_values = 0b, 1b, 2b ; temperature_sd_test:flag_meanings = "passed failed unknown" ; temperature_sd_test:coordinates = "time latitude longitude pressure" ; double salinity(pressure) ; salinity:_FillValue = -99.99 ; salinity:standard_name = "sea_water_salinity" ; salinity:units = "psu" ; salinity:valid_min = 0 ; salinity:valid_max = 45 ;salinity:ancillary_variables = "salinity_whole_profile_flag salinity_qc_flag salinity_sd_test" salinity:coordinates = "time latitude longitude pressure" ;
byte salinity_whole_profile_flag ;
salinity_whole_profile_flag:long_name = "qc flag for whole salinity
profile (primary L1 flag)" ;
salinity_whole_profile_flag:quality_control_convention = "IODE qc
scheme March 2012" ;
salinity_whole_profile_flag:valid_min = 1 ;
salinity_whole_profile_flag:valid_max = 9 ;
salinity_whole_profile_flag:flag_values = 1b, 2b, 3b, 4b, 9b ;
salinity_whole_profile_flag:flag_meanings = "good
not_evaluated_or_unknown suspect bad missing" ;
byte salinity_qc_flag(pressure) ;
salinity_qc_flag:long_name = "quality control flag for salinity
(primary Level 1 flag)" ;
salinity_qc_flag:standard_name = "sea_water_salinity
status_flag" ;
salinity_qc_flag:quality_control_convention = "Proposed IODE qc
scheme March 2012" ;
salinity_qc_flag:valid_min = 1 ;
salinity_qc_flag:valid_max = 9 ;
salinity_qc_flag:flag_values = 1b, 2b, 3b, 4b, 9b ;
salinity_qc_flag:flag_meanings = "good not_evaluated_or_unknown
suspect bad missing" ;
salinity_qc_flag:coordinates = "time latitude longitude
pressure" ;
byte salinity_sd_test(pressure) ;
salinity_sd_test:long_name = "qc flag for monthly salinity 3
standard deviation test (secondary L2 flag)"
salinity_sd_test:quality_control_convention = "IODE qc scheme March
2012" ;
salinity_sd_test:valid_min = 0 ;
salinity_sd_test:valid_max = 2 ;
salinity_sd_test:flag_values = 0b, 1b, 2b ;
salinity_sd_test:flag_meanings = "passed failed unknown" ;
salinity_sd_test:coordinates = "time latitude longitude
pressure" ;
int profile ; //Unique integer to identify each profile
profile:long_name = "profile identifier"
profile:cf_role = "profile_id";
// global attributes:
:conventions = "CF-1.6" ;
:featureType = "profile" ;
:cdm_data_type = "Profile" ;
data:
time = 22509.0081018517 ;
time_qc_flag = 1;
latitude = -10.5688 ;
longitude = 142.11713 ;
position_qc_flag = 1;
pressure = 4, 5, 6, 7, 8, 9, 10, 11, 12 ;
temperature = 24.737, 24.737, 24.739, 24.738, 24.737, 24.736, 24.7375,
24.737, 24.735 ;
temperature_whole_profile_flag = 1;
temperature_qc_flag = 1,1,1,1,1,1,1,1,1;
temperature_sd_test = 0,0,0,0,0,0,0,0,0;
salinity = 35.2028, 35.2032, 35.2013, 35.2018, 35.2022, 35.2019, 35.2019,
35.2024, 35.204 ;
salinity_whole_profile_flag = 1;
salinity_qc_flag = 1,1,1,1,1,1,1,1,1;
salinity_sd_test = 0,0,0,0,0,0,0,0,0;
profile = 1234567 ;
_______________________________________________ CF-metadata mailing list [email protected] http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
