I agree. Mean bias is more useful for model evaluation. This could of course be applied also to mean winter bias, or mean summer bias not just annual.
Paul From: <Boustani>, "Maziyar (398F)" <maziyar.boust...@jpl.nasa.gov<mailto:maziyar.boust...@jpl.nasa.gov>> Date: Wednesday, June 4, 2014 4:41 PM To: Paul Loikith <paul.c.loik...@jpl.nasa.gov<mailto:paul.c.loik...@jpl.nasa.gov>>, Alex Goodman <alex.good...@colostate.edu<mailto:alex.good...@colostate.edu>> Cc: "dev@climate.apache.org<mailto:dev@climate.apache.org>" <dev@climate.apache.org<mailto:dev@climate.apache.org>> Subject: Re: Model Observation Evaluation exmaple I guess the best solution here is to add the new Metric "calcBiasAberageOverTime" to our Metrics and use that in this example to be able to generate one plot. …………………… Maziyar Boustani Software Engineer - GIS Developer Science Data Understanding Group Jet Propulsion Laboratory, California Institute of Technology 4800 Oak Grove Drive, Pasadena, CA, 91109 maziyar.boust...@jpl.nasa.gov<mailto:maziyar.boust...@jpl.nasa.gov> On Jun 4, 2014, at 4:29 PM, Loikith, Paul C (329C-Affiliate) wrote: Yes agreed. I'm guessing the issue here is the use of "calcBias" vs. "calcBiasAveragedOverTime." The later module from metrics.py would accomplish this. I definitely agree about the colorbar too. Hope all is well. Paul From: Alex Goodman <alex.good...@colostate.edu<mailto:alex.good...@colostate.edu>> Date: Wednesday, June 4, 2014 4:25 PM To: "dev@climate.apache.org<mailto:dev@climate.apache.org>" <dev@climate.apache.org<mailto:dev@climate.apache.org>> Cc: "Boustani, Maziyar (398F)" <maziyar.boust...@jpl.nasa.gov<mailto:maziyar.boust...@jpl.nasa.gov>>, Paul Loikith <paul.c.loik...@jpl.nasa.gov<mailto:paul.c.loik...@jpl.nasa.gov>> Subject: Re: Model Observation Evaluation exmaple Hi Paul, I think I wasn't sure what was exactly in Mazi plots. I thought what was being shown was the bias averaged for each year. Instead it sounds like the data was temporally regridded from monthly data to yearly data first, (eg, the annual average surface temperatures were calculated first, then the bias metric was ran on the temporally regridded data). What I was trying to say was that from visually inspecting the plots alone, a plot of the average values of all five of them would probably look similar to each individual plot. Sorry if I worded that poorly... in any case I think your suggestion is the correct one, what should be plotted is just the mean bias at each grid point for the five-year period. Also another side-note: When plotting biases or anomalies, it is generally ideal to have the 0 contour at the center of the colorbar. Currently you must manually enter the contour levels into one of the arguments of the contour plotting function to achieve this. Finding an elegant way to do this automatically was an often requested feature of the OCW plotting interface that I unfortunately never found the time to implement. Just something to think about. Also, thanks for the kind regards Chris, I am doing well. Alex On Wed, Jun 4, 2014 at 5:06 PM, Mattmann, Chris A (3980) <chris.a.mattm...@jpl.nasa.gov<mailto:chris.a.mattm...@jpl.nasa.gov>> wrote: THanks guys, Alex, great to see you! ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chris Mattmann, Ph.D. Chief Architect Instrument Software and Science Data Systems Section (398) NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 168-519, Mailstop: 168-527 Email: chris.a.mattm...@nasa.gov<mailto:chris.a.mattm...@nasa.gov> WWW: http://sunset.usc.edu/~mattmann/ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Adjunct Associate Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -----Original Message----- From: Alex Goodman <alex.good...@colostate.edu<mailto:alex.good...@colostate.edu>> Reply-To: "dev@climate.apache.org<mailto:dev@climate.apache.org>" <dev@climate.apache.org<mailto:dev@climate.apache.org>> Date: Wednesday, June 4, 2014 3:53 PM To: "Boustani, Maziyar (398F)" <maziyar.boust...@jpl.nasa.gov<mailto:maziyar.boust...@jpl.nasa.gov>> Cc: "Loikith, Paul C (329C-Affiliate)" <paul.c.loik...@jpl.nasa.gov<mailto:paul.c.loik...@jpl.nasa.gov>>, "dev@climate.apache.org<mailto:dev@climate.apache.org>" <dev@climate.apache.org<mailto:dev@climate.apache.org>> Subject: Re: Model Observation Evaluation exmaple >Hi Mazi, > >I am in agreement with Paul. It is also clear from a quick inspection of >your plots that the model bias for each year is identical, so plotting >just >the 5-year average bias would give you sufficient information. You might >want to take a quick glance at IPCC AR5 WG I, Ch. 9 [1] for some good >examples. > >Thanks, >Alex > >[1] - >http://www.climatechange2013.org/images/report/WG1AR5_Chapter09_FINAL.pdf > > >On Wed, Jun 4, 2014 at 4:48 PM, Boustani, Maziyar (398F) < >maziyar.boust...@jpl.nasa.gov<mailto:maziyar.boust...@jpl.nasa.gov>> wrote: > >> Here is the attached [1]. >> >> [1]: http://oi58.tinypic.com/w7yo2q.jpg >> >> ŠŠŠŠŠŠŠŠ >> Maziyar Boustani >> Software Engineer - GIS Developer >> Science Data Understanding Group >> Jet Propulsion Laboratory, California Institute of Technology >> 4800 Oak Grove Drive, >> Pasadena, CA, 91109 >> maziyar.boust...@jpl.nasa.gov<mailto:maziyar.boust...@jpl.nasa.gov> >> >> On Jun 4, 2014, at 3:35 PM, Loikith, Paul C (329C-Affiliate) wrote: >> >> Hi Mazi, >> >> This should be one plot total. When you compute a bias, conventionally >> it's just one bias for the entire time period so there should be one >>plot. >> You could have multiple plots if you were computing the bias for all 4 >> seasons, but for annual there should only be one. >> >> When you say you temporally regridded to 365 days, what do you mean by >> this? The model and obs data are monthly, correct? So shouldn't you >>just >> be computing the mean difference between the obs and the model? >> >> Please let me know if you need further clarification. >> >> Thanks, >> >> Paul >> >> >> >> From: <Boustani>, "Maziyar (398F)" >> <maziyar.boust...@jpl.nasa.gov<mailto:maziyar.boust...@jpl.nasa.gov>> >> Date: Wednesday, June 4, 2014 3:12 PM >> To: Paul Loikith >> <paul.c.loik...@jpl.nasa.gov<mailto:paul.c.loik...@jpl.nasa.gov>>, >> "good...@apache.org<mailto:good...@apache.org>" < >> good...@apache.org<mailto:good...@apache.org>> >> Cc: "dev@climate.apache.org<mailto:dev@climate.apache.org>" >> <dev@climate.apache.org<mailto:dev@climate.apache.org>> >> Subject: Model Observation Evaluation exmaple >> >> Hi Paul and Alex, >> >> I ran a test code to evaluate one model [1] with one observation[2]. >>You >> can find the result as attached. >> This has been temporally regrided for 356 day, so we going to have 5 >>years >> of data and that is why we have 5 plots. >> Do you guys thinks this is correct to have one plot for each time or it >> should be one plot for entire time? >> >> Metric = Bias >> parameter_id = 38 >> dataset_id = 10 >> min_lat = 0 >> max_lat = 20 >> min_lon = 0 >> max_lon = 20 >> start_time = 2000-01-01 00:00:00 >> end_time = 2004-12-01 00:00:00 >> >> >> [1] : AFRICA_UQAM-CRCM5_CTL_ERAINT_MM_50km_1989-2008_tas.nc (variable = >> TAS) >> [2] : CRU3.1 Daily-Mean Temperature (parmeter_id = 38) >> >> <model_vs_obs_bias.png> >> >> ŠŠŠŠŠŠŠŠ >> Maziyar Boustani >> Software Engineer - GIS Developer >> Science Data Understanding Group >> Jet Propulsion Laboratory, California Institute of Technology >> 4800 Oak Grove Drive, >> Pasadena, CA, 91109 >> maziyar.boust...@jpl.nasa.gov<mailto:maziyar.boust...@jpl.nasa.gov> >> >> <model_vs_obs_bias.png> >> >> >> > > >-- >Alex Goodman >Graduate Research Assistant >Department of Atmospheric Science >Colorado State University -- Alex Goodman Graduate Research Assistant Department of Atmospheric Science Colorado State University
