Hi all, It's working now and I'm getting results using RCMs and GCMs vs TRMM, CRU ... Now I've two questions : - Is it possible to compute PDFs and CDFs and other metrics? - Is it possible to compute these metrics on other parameters such as 2m temperature, 500mb height, wind at different levels to compare with reanalysis for example?
Best regards Rashyd 2014-06-01 12:50 GMT-04:00 Kim, Jinwon <[email protected]>: > The problem was the correct rcmet.py was not copied to > "rcmet2.1.3/src/main/python/bin". Copying rcmet.py in > "rcmet2.1.3/src/main/python/rcmes" into "rcmet2.1.3/src/main/python/bin" > will resolve this problem. I've always run rcmet from > "rcmet2.1.3/src/main/python/rcmes" and did not recognize this until today. > > Will you please replace rcmet.py in "rcmet2.1.3/src/main/python/bin" with > the one on "rcmet2.1.3/src/main/python/rcmes"? thanks. > > ----------------------------------------------------------------- > Jinwon Kim > Joint Institute for Regional Earth System Science and Engineering and > Dept. Atmospheric and Oceanic Sciences > University of California, Los Angeles > 607 Charles E Young Drive East, Young Hall, Room 4242, Los Angeles, CA > 90095-7228, USA > [email protected]; 310-206-2828 > ________________________________________ > From: Mattmann, Chris A (3980) [[email protected]] > Sent: Sunday, June 01, 2014 9:43 AM > To: Rashyd ZAABOUL; Kim, Jinwon > Cc: [email protected]; [email protected]; Joyce, Michael J > (398J); rcmes-dev; Goodale, Cameron E (398J); [email protected]; > Mcgibbney, Lewis J (398J) > Subject: Re: failed to run rcmes following the step-by-step document > > Hi Rashyd, > > I'm CC'ing the Apache list for the Open Climate Workbench project where > we have a lot of eyes on the list. Maybe someone on the Apache list can > help as well. > > Also we are happy to have a Skype/telecon with you this week to get > your code working. Cam, Mike or Lewis or Mazi or Shakeh - can you guys > work with Rashyd to get his problem fixed. > > Jinwon -- can you please let Cam, Mike or Lewis or Mazi or Shakeh, or > Paul help out on this - they'll take it from here. > > Cheers, > Chris > > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > Chris Mattmann, Ph.D. > Chief Architect > Instrument Software and Science Data Systems Section (398) > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA > Office: 168-519, Mailstop: 168-5th floor > Email: [email protected] > WWW: http://sunset.usc.edu/~mattmann/ > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > Adjunct Associate Professor, Computer Science Department > University of Southern California, Los Angeles, CA 90089 USA > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > > > > > > -----Original Message----- > From: Rashyd ZAABOUL <[email protected]> > Date: Sunday, June 1, 2014 2:31 AM > To: "Kim, Jinwon (329M-Affiliate)" <[email protected]> > Cc: "[email protected]" <[email protected]>, > "[email protected]" <[email protected]>, "Joyce, Michael J > (398J)" <[email protected]>, rcmes-dev > <[email protected]>, "Goodale, Cameron E (398J)" > <[email protected]> > Subject: Re: failed to run rcmes following the step-by-step document > > >Hi Kim,What do you mean by all files are OK? the rcmet.py is not OK! it > >contains params = db.getParams() on line 194 !!! > > > > > >I've tried this tar file which is equal to the previous one (I've > >modified the line 194 in rcmet.py) and I've got the same error: > > > > > >File "rcmet.py <http://rcmet.ok.py/>", line 237, in runUsingConfig > > > >(jobProperties,obsSource,obsDatasetList,obsList,obsVarName,obsLonName,obsL > >atName,obsTimeName,obsTimestep,gridBox,models) > >ValueError: too many values to unpack > > > > > > > > > >Regards > >Rashyd > > > > > > > >2014-06-01 11:26 GMT+04:00 Kim, Jinwon <[email protected]>: > > > >Hmm, I checked the tar file I sent - all files are OK. I'm sending the > >tar file again. Please check this one out. Please clean up previous > >versions and try the attached one with the included config file > >(cordexMENA.cfg). This is the code/config file I used > > to create the figures I sent in the earlier e-mail. > > > >For your error messages: > > > >(1) The parameter "obsSource" directs the code where to find observation > >data for evaluation. "obsSource=1" is reserved for the case users provide > >their own observation data. If you set obsSource=1 and do not provide > >data, it will result in error exit as the > > code cannot find the observation data file(s) specified by the user. > >Thus, unless you provide your own observation data, obsSource=1 must not > >be used. > > > >(2) The exit message "No input observation data file. EXIT" occurs when > >there is no observation data file id's are assigned when obsSource=0 is > >set. This can happen if "obsParamId" in the config file is not properly > >set. We don't expect any problems if obsParamId > > is correctly set because the data specified by obsParamId are included > >in the database (this requires active internet connection). Will you send > >me your configure file so that I can look at it? > > > > > > > > > >Thanks for your patience. > > > > > >----------------------------------------------------------------- > >Jinwon Kim > >Joint Institute for Regional Earth System Science and Engineering and > >Dept. Atmospheric and Oceanic Sciences > >University of California, Los Angeles > >607 Charles E Young Drive East, Young Hall, Room 4242, Los Angeles, CA > >90095-7228, USA > >[email protected]; 310-206-2828 > >________________________________________ > >From: Rashyd ZAABOUL [[email protected]] > > > >Sent: Saturday, May 31, 2014 11:58 PM > >To: Kim, Jinwon > >Cc: [email protected]; > >[email protected] <mailto:[email protected]>; Joyce, Michael J > >(398J); rcmes-dev; > >[email protected] <mailto:[email protected]> > >Subject: Re: failed to run rcmes following the step-by-step document > > > > > >Hi Kim, > > > >Thank you for your help. I'm sorry but I'm getting confused with all > >these different packages I received from you and your colleagues!. The > >last one you just sent to me still contains params = db.getParams() in > >rcmet.py at line 194. I've replaced this file > > with the one sent by Cameron before and it seems to be ok. But when I've > >tried to reproduce the result you sent to me (with obsSource=1) I got > >this error : > > > >... > >data_prep is completed: both obs and mdl data are re-gridded to a common > >analysis grid > >Traceback (most recent call last): > > File "rcmet.py", line 299, in <module> > > runUsingConfig(args.CONFIG) > > File "rcmet.py", line 237, in runUsingConfig > > > >(jobProperties,obsSource,obsDatasetList,obsList,obsVarName,obsLonName,obsL > >atName,obsTimeName,obsTimestep,gridBox,models) > >ValueError: too many values to unpack > > > >Do you know how can I fix this please? > > > >NOTE: with obsSource=0 I got : > >No input observation data file. EXIT > > > >Thanks and regards > >Rashyd > > > > > > > >2014-05-29 20:08 GMT+04:00 Kim, Jinwon > ><[email protected]<mailto:[email protected]>>: > >To all, > > > >*** Please update your RCMET with the attached version 2.1.3. *** > > > >I found that some files included in the earlier tar file for the release > >version 2.1.3 have some critical issues. Now I've applied all fixes and > >successfully ran a test case with a MENA domain dataset; the version > >works properly now. I apologize for inconvenience. > > > >Rashyd: > >Please see the two attached figures for the MENA domain (model bias, GPCP > >climatology for 1991-2000 over the MENA domain) for your reference. The > >configure file for this processing is: > >"rcmet2.1.3/src/main/python/rcmes/cordexcordexMENA.cfg". The analysis > >domain has been adjusted to fit the MENA domain. > > > > > >Aanjay and Rama: > >The code also works fine with the India domain. The two attached figures > >for the India domain shows the TRMM climatology for 98-2000 and the bias > >of model ensemble against TRMM for the same period. The sample configure > >file is: cordexIndia.cfg under resources/ > > (the same location as the MENA config file). > > > >Please contact me/mike/rcmes-dev if you have any questions. > > > > > >With best regards, > >----------------------------------------------------------------- > >Jinwon Kim > >Joint Institute for Regional Earth System Science and Engineering and > >Dept. Atmospheric and Oceanic Sciences > >University of California, Los Angeles > >607 Charles E Young Drive East, Young Hall, Room 4242, Los Angeles, CA > >90095-7228, USA > > > >[email protected]<mailto:[email protected]>; 310-206-2828 > >________________________________________ > >From: Rashyd ZAABOUL [[email protected]<mailto:[email protected]>] > > > >Sent: Thursday, May 29, 2014 12:07 AM > >To: Kim, Jinwon > >Cc: Joyce, Michael J (398J); rcmes-dev > >Subject: Re: failed to run rcmes following the step-by-step document > > > >Hi Michael, > >The package you've just sent to me contains the same rcmet.py as the > >package 2.1.2 and has on line 194 : > > params = db.getParams() > > > >Is it the right one? Because I'm still having the same error : > > > >Running using config file: ../rcmes/resources/cordexAF.cfg > >Traceback (most recent call last): > > File "rcmet.py", line 299, in <module> > > runUsingConfig(args.CONFIG) > > File "rcmet.py", line 194, in runUsingConfig > > params = db.getParams() > >AttributeError: 'module' object has no attribute 'getParams' > > > >I've attached one of the files that I'm trying to use. > > > >Thanks again > >Rashyd > > > > > > > > > >2014-05-29 10:29 GMT+04:00 Kim, Jinwon > ><[email protected]<mailto:[email protected]><mailto: > [email protected] > >u<mailto:[email protected]>>>: > >Here comes the attachment. > > > >----------------------------------------------------------------- > >Jinwon Kim > >Joint Institute for Regional Earth System Science and Engineering and > >Dept. Atmospheric and Oceanic Sciences > >University of California, Los Angeles > >607 Charles E Young Drive East, Young Hall, Room 4242, Los Angeles, CA > >90095-7228, USA > > > >[email protected]<mailto:[email protected]><mailto: > [email protected] > ><mailto:[email protected]>>; > > 310-206-2828 > >________________________________________ > >From: Rashyd ZAABOUL > >[[email protected]<mailto:[email protected]><mailto:[email protected] > <mail > >to:[email protected]>>] > > > >Sent: Wednesday, May 28, 2014 10:27 PM > >To: Joyce, Michael J (398J) > >Cc: Kim, Jinwon; rcmes-dev > >Subject: Re: failed to run rcmes following the step-by-step document > > > >Hi Michael, > > > >With the new Jinwon's RCMET version 2.1.3 I got this error : > > > >Running using config file: ../rcmes/resources/cordexAF.cfg > >Traceback (most recent call last): > > File "rcmet.py", line 299, in <module> > > runUsingConfig(args.CONFIG) > > File "rcmet.py", line 194, in runUsingConfig > > params = db.getParams() > >AttributeError: 'module' object has no attribute 'getParams' > > > >My > >PYTHONPATH=/home/wrf/rcmet-2.1.3:/home/wrf/rcmet-2.1.3/src/main/python/rcm > >es > > > >I'm using files for precipitation exactly as I download them from ESGF > >portal for CORDEX MENA domain. The files I'm using are : > > > >pr_MNA-44_CNRM-CERFACS-CNRM-CM5_historical_r1i1p1_SMHI-RCA4_v1_mon_197101- > >198012.nc > >pr_MNA-44_CNRM-CERFACS-CNRM-CM5_historical_r1i1p1_SMHI-RCA4_v1_mon_198101- > >199012.nc > >pr_MNA-44_CNRM-CERFACS-CNRM-CM5_historical_r1i1p1_SMHI-RCA4_v1_mon_199101- > >200012.nc > >pr_MNA-44_CNRM-CERFACS-CNRM-CM5_historical_r1i1p1_SMHI-RCA4_v1_mon_200101- > >200512.nc > >pr_MNA-44_ICHEC-EC-EARTH_historical_r12i1p1_SMHI-RCA4_v1_mon_197101-198012 > >.nc > >pr_MNA-44_ICHEC-EC-EARTH_historical_r12i1p1_SMHI-RCA4_v1_mon_198101-199012 > >.nc > >pr_MNA-44_ICHEC-EC-EARTH_historical_r12i1p1_SMHI-RCA4_v1_mon_199101-200012 > >.nc > >pr_MNA-44_ICHEC-EC-EARTH_historical_r12i1p1_SMHI-RCA4_v1_mon_200101-200512 > >.nc > >pr_MNA-44_NOAA-GFDL-GFDL-ESM2M_historical_r1i1p1_SMHI-RCA4_v1_mon_197101-1 > >98012.nc > >pr_MNA-44_NOAA-GFDL-GFDL-ESM2M_historical_r1i1p1_SMHI-RCA4_v1_mon_198101-1 > >99012.nc > >pr_MNA-44_NOAA-GFDL-GFDL-ESM2M_historical_r1i1p1_SMHI-RCA4_v1_mon_199101-2 > >00012.nc > >pr_MNA-44_NOAA-GFDL-GFDL-ESM2M_historical_r1i1p1_SMHI-RCA4_v1_mon_200101-2 > >00512.nc > > > > > >Do I have to pre-process this data first before I can use it with RCMET ? > > > >I've attached my config file to check if I'm missing something. > > > >Thanks for your help > > > >Rashyd > > > > > > > > > >2014-05-28 18:05 GMT+04:00 Joyce, Michael J (398J) > ><[email protected]<mailto:[email protected] > ><mailto: > >[email protected]<mailto:[email protected] > >><mailto: > >[email protected]<mailto:[email protected]><mailto: > M > >[email protected]<mailto:[email protected]>>>>: > >Hi Rashyd, > > > >It looks like your¹e using one of Jinwon¹s RCMET versions. He recently > >released a newer version that may resolve some of your problems. You can > >find and download it at [1]. I¹m sure Jinwon will be able to help you get > >this resolved. Thank you for sending the > > document you used. I¹ll be sure to get it and the website updated with > >the correct information. > > > >[1] > >https://github.com/apache/climate/tree/jinwon-rcmet-2.1.3 > ><https://github.com/apache/climate/tree/jinwon-rcmet-2.1.3> > > > >Thanks much! > > > >-- > >Michael J. Joyce > >Engineering Applications Software Engineer > >Instrument Software and Science Data Systems > >NASA Jet Propulsion Laboratory > >California Institute of Technology > >4800 Oak Grove Drive > >Pasadena, California 91109 > >Mail Stop: 158-242 > >Tel: (818) 354-7550 > >Fax: (818) 393-1370 > > > > > > > >From: Rashyd ZAABOUL > ><[email protected]<mailto:[email protected]><mailto:[email protected] > <mail > >to:[email protected]>><mailto:[email protected]<mailto:[email protected] > >< > >mailto:[email protected]<mailto:[email protected]>>>> > >Date: Wednesday, May 28, 2014 at 5:09 AM > > > >To: "Kim, Jinwon (329M-Affiliate)" > ><[email protected]<mailto:[email protected]><mailto: > [email protected] > >u<mailto:[email protected]>><mailto:[email protected]<mailto: > jkim@atmo > >s.ucla.edu><mailto:[email protected]<mailto:[email protected]>>>> > >Cc: rcmes-dev > ><[email protected]<mailto:[email protected]><mailto: > rcmes-dev@jp > >l.nasa.gov<mailto:[email protected]>><mailto:[email protected] > <m > >ailto:[email protected]><mailto:[email protected]<mailto: > rcmes-d > >[email protected]>>>>, > > JPL > ><[email protected]<mailto:[email protected] > ><mailto: > >[email protected]<mailto:[email protected] > >><mailto: > >[email protected]<mailto:[email protected]><mailto: > M > >[email protected]<mailto:[email protected]>>>> > >Subject: Re: failed to run rcmes following the step-by-step document > > > >Hi Jim and Michael, > > > >I've modified the script > >~rcmet-2.1.2/src/main/python/rcmes/storage/rcmed.py by replacing > >json.loads function by simplejson.loads one to get more details on the > >error and this is what I got : > > > >Running using config file: ../rcmes/resources/mena.cfg > >Traceback (most recent call last): > > File "rcmet.py", line 299, in <module> > > runUsingConfig(args.CONFIG) > > File "rcmet.py", line 194, in runUsingConfig > > params = db.getParams() > > File "/home/rcmes/rcmet-2.1.2/src/main/python/rcmes/storage/rcmed.py", > >line 27, in getParams > > paramDict = simplejson.loads(httpRequest.text) > > File > >"/home/rcmes/anaconda/lib/python2.7/site-packages/simplejson-3.5.2-py2.7-l > >inux-x86_64.egg/simplejson/__init__.py", line 501, in loads > > return _default_decoder.decode(s) > > File > >"/home/rcmes/anaconda/lib/python2.7/site-packages/simplejson-3.5.2-py2.7-l > >inux-x86_64.egg/simplejson/decoder.py", line 370, in decode > > obj, end = self.raw_decode(s) > > File > >"/home/rcmes/anaconda/lib/python2.7/site-packages/simplejson-3.5.2-py2.7-l > >inux-x86_64.egg/simplejson/decoder.py", line 389, in raw_decode > > return self.scan_once(s, idx=_w(s, idx).end()) > >simplejson.scanner.JSONDecodeError: Expecting value: line 1 column 1 > >(char 0) > > > >What does this error mean ? I'm afraid the database RCMED is down while > >I'm trying to get data out of it ! > > > >Best regards > >Rashyd > > > > > > > > > >2014-05-27 8:13 GMT-07:00 Kim, Jinwon > ><[email protected]<mailto:[email protected]><mailto: > [email protected] > >u<mailto:[email protected]>><mailto:[email protected]<mailto: > jkim@atmo > >s.ucla.edu><mailto:[email protected]<mailto:[email protected]>>>>: > >Thanks for your interests. > > > >For organizing your data: > > > >You can put your model data in your own directory. Then specify the link > >to the data (or data files) in the configure file, [MODEL] block. The > >code currently takes netcdf format files for input. > > > >If you use the observed data in the RCMES database, just specify the > >parameter numbers in the [RCMED] block, also in the configure file. If > >you use your own observation data, you can specify it in the same block > >(it also takes netcdf-format files). > > > >Hope this helps. > > > > > >With best regards, > > > >----------------------------------------------------------------- > >Jinwon Kim > >Joint Institute for Regional Earth System Science and Engineering and > >Dept. Atmospheric and Oceanic Sciences > >University of California, Los Angeles > >607 Charles E Young Drive East, Young Hall, Room 4242, Los Angeles, CA > >90095-7228, USA > > > >[email protected]<mailto:[email protected]><mailto: > [email protected] > ><mailto:[email protected]>><mailto:[email protected]<mailto: > jkim@atmos > >.ucla.edu><mailto:[email protected]<mailto:[email protected]>>>; > > 310-206-2828 > >________________________________________ > >From: Rashyd ZAABOUL > >[[email protected]<mailto:[email protected]><mailto:[email protected] > <mail > >to:[email protected]>><mailto:[email protected]<mailto:[email protected] > >< > >mailto:[email protected]<mailto:[email protected]>>>] > >Sent: Tuesday, May 27, 2014 7:59 AM > > > >To: > >[email protected]<mailto:[email protected]><mailto: > rcmes-dev@jpl > >.nasa.gov<mailto:[email protected]>><mailto:[email protected] > <ma > >ilto:[email protected]><mailto:[email protected]<mailto: > rcmes-de > >[email protected]>>> > >Subject: failed to run rcmes following the step-by-step document > > > >Hi > >I'm trying to use RCMES to evaluate CORDEX models outputs against CRU, > >TRMM ... I followed the step-by-step document but it seems that something > >is missing there. For example it doesn't tell how to organize my data? > >where to put the observations ... > > > >Could you please provide me with any useful documentation that can help > >me setup RCMES to evaluate those models? > > > >Thanks > >Rashyd > > > >-- > >Rashyd Zaaboul > >Climate and Water Modeling Scientist > >International Center for Biosaline Agriculture > >P.O. Box 14660 Dubai, UAE > >Mobile : +971 50 675 2007 > >Office : +971 4 336 1100 x 259 > > > >[email protected]<mailto:[email protected]><mailto: > r.zaa > >[email protected]<mailto:[email protected]>><mailto: > r.zaaboul > >@biosaline.org.ae<mailto:[email protected]><mailto: > r.zaaboul@bios > >aline.org.ae<mailto:[email protected]>>><mailto: > r.zaaboul@biosali > >ne.org.ae<mailto:[email protected]><mailto: > [email protected] > >g.ae<mailto:[email protected]>><mailto: > [email protected] > ><mailto:[email protected]><mailto:[email protected] > <mail > >to:[email protected]>>>> > > > >Please consider the environment before printing this e-mail > > > > > > > >-- > >Rashyd Zaaboul > >Climate and Water Modeling Scientist > >International Center for Biosaline Agriculture > >P.O. Box 14660 Dubai, UAE > >Mobile : +971 50 675 2007 > >Office : +971 4 336 1100 x 259 > >[email protected]<mailto:[email protected]><mailto: > r.zaa > >[email protected]<mailto:[email protected]>><mailto: > r.zaaboul > >@biosaline.org.ae<mailto:[email protected]><mailto: > r.zaaboul@bios > >aline.org.ae<mailto:[email protected]>>> > > > >Please consider the environment before printing this e-mail > > > > > > > >-- > >Rashyd Zaaboul > >Climate and Water Modeling Scientist > >International Center for Biosaline Agriculture > >P.O. Box 14660 Dubai, UAE > >Mobile : +971 50 675 2007 > >Office : +971 4 336 1100 x 259 > >[email protected]<mailto:[email protected]><mailto: > r.zaa > >[email protected]<mailto:[email protected]>><mailto: > r.zaaboul > >@biosaline.org.ae<mailto:[email protected]><mailto: > r.zaaboul@bios > >aline.org.ae<mailto:[email protected]>>> > > > >Please consider the environment before printing this e-mail > > > > > > > >-- > >Rashyd Zaaboul > >Climate and Water Modeling Scientist > >International Center for Biosaline Agriculture > >P.O. Box 14660 Dubai, UAE > >Mobile : +971 50 675 2007 > >Office : +971 4 336 1100 x 259 > >[email protected]<mailto:[email protected]><mailto: > r.zaa > >[email protected]<mailto:[email protected]>> > > > >Please consider the environment before printing this e-mail > > > > > > > >-- > >Rashyd Zaaboul > >Climate and Water Modeling Scientist > >International Center for Biosaline Agriculture > >P.O. Box 14660 Dubai, UAE > >Mobile : +971 50 675 2007 > >Office : +971 4 336 1100 x 259 > >[email protected]<mailto:[email protected]> > > > >Please consider the environment before printing this e-mail > > > > > > > > > > > > > > > > > >-- > >Rashyd Zaaboul > > > >Climate and Water Modeling Scientist > > > >International Center for Biosaline Agriculture > >P.O. Box 14660 Dubai, UAE > >Mobile : +971 50 675 2007 > >Office : +971 4 336 1100 x 259 > > > >[email protected] > > > >Please consider the environment before printing this e-mail > > > > > > -- Rashyd Zaaboul *Climate and Water Modeling Scientist* International Center for Biosaline Agriculture P.O. Box 14660 Dubai, UAE Mobile : +971 50 675 2007 Office : +971 4 336 1100 x 259 [email protected] *Please consider the environment before printing this e-mail*
