The problem was the correct rcmet.py was not copied to "rcmet2.1.3/src/main/python/bin". Copying rcmet.py in "rcmet2.1.3/src/main/python/rcmes" into "rcmet2.1.3/src/main/python/bin" will resolve this problem. I've always run rcmet from "rcmet2.1.3/src/main/python/rcmes" and did not recognize this until today.
Will you please replace rcmet.py in "rcmet2.1.3/src/main/python/bin" with the one on "rcmet2.1.3/src/main/python/rcmes"? thanks. ----------------------------------------------------------------- Jinwon Kim Joint Institute for Regional Earth System Science and Engineering and Dept. Atmospheric and Oceanic Sciences University of California, Los Angeles 607 Charles E Young Drive East, Young Hall, Room 4242, Los Angeles, CA 90095-7228, USA [email protected]; 310-206-2828 ________________________________________ From: Mattmann, Chris A (3980) [[email protected]] Sent: Sunday, June 01, 2014 9:43 AM To: Rashyd ZAABOUL; Kim, Jinwon Cc: [email protected]; [email protected]; Joyce, Michael J (398J); rcmes-dev; Goodale, Cameron E (398J); [email protected]; Mcgibbney, Lewis J (398J) Subject: Re: failed to run rcmes following the step-by-step document Hi Rashyd, I'm CC'ing the Apache list for the Open Climate Workbench project where we have a lot of eyes on the list. Maybe someone on the Apache list can help as well. Also we are happy to have a Skype/telecon with you this week to get your code working. Cam, Mike or Lewis or Mazi or Shakeh - can you guys work with Rashyd to get his problem fixed. Jinwon -- can you please let Cam, Mike or Lewis or Mazi or Shakeh, or Paul help out on this - they'll take it from here. Cheers, Chris ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chris Mattmann, Ph.D. Chief Architect Instrument Software and Science Data Systems Section (398) NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 168-519, Mailstop: 168-5th floor Email: [email protected] WWW: http://sunset.usc.edu/~mattmann/ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Adjunct Associate Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -----Original Message----- From: Rashyd ZAABOUL <[email protected]> Date: Sunday, June 1, 2014 2:31 AM To: "Kim, Jinwon (329M-Affiliate)" <[email protected]> Cc: "[email protected]" <[email protected]>, "[email protected]" <[email protected]>, "Joyce, Michael J (398J)" <[email protected]>, rcmes-dev <[email protected]>, "Goodale, Cameron E (398J)" <[email protected]> Subject: Re: failed to run rcmes following the step-by-step document >Hi Kim,What do you mean by all files are OK? the rcmet.py is not OK! it >contains params = db.getParams() on line 194 !!! > > >I've tried this tar file which is equal to the previous one (I've >modified the line 194 in rcmet.py) and I've got the same error: > > >File "rcmet.py <http://rcmet.ok.py/>", line 237, in runUsingConfig > >(jobProperties,obsSource,obsDatasetList,obsList,obsVarName,obsLonName,obsL >atName,obsTimeName,obsTimestep,gridBox,models) >ValueError: too many values to unpack > > > > >Regards >Rashyd > > > >2014-06-01 11:26 GMT+04:00 Kim, Jinwon <[email protected]>: > >Hmm, I checked the tar file I sent - all files are OK. I'm sending the >tar file again. Please check this one out. Please clean up previous >versions and try the attached one with the included config file >(cordexMENA.cfg). This is the code/config file I used > to create the figures I sent in the earlier e-mail. > >For your error messages: > >(1) The parameter "obsSource" directs the code where to find observation >data for evaluation. "obsSource=1" is reserved for the case users provide >their own observation data. If you set obsSource=1 and do not provide >data, it will result in error exit as the > code cannot find the observation data file(s) specified by the user. >Thus, unless you provide your own observation data, obsSource=1 must not >be used. > >(2) The exit message "No input observation data file. EXIT" occurs when >there is no observation data file id's are assigned when obsSource=0 is >set. This can happen if "obsParamId" in the config file is not properly >set. We don't expect any problems if obsParamId > is correctly set because the data specified by obsParamId are included >in the database (this requires active internet connection). Will you send >me your configure file so that I can look at it? > > > > >Thanks for your patience. > > >----------------------------------------------------------------- >Jinwon Kim >Joint Institute for Regional Earth System Science and Engineering and >Dept. Atmospheric and Oceanic Sciences >University of California, Los Angeles >607 Charles E Young Drive East, Young Hall, Room 4242, Los Angeles, CA >90095-7228, USA >[email protected]; 310-206-2828 >________________________________________ >From: Rashyd ZAABOUL [[email protected]] > >Sent: Saturday, May 31, 2014 11:58 PM >To: Kim, Jinwon >Cc: [email protected]; >[email protected] <mailto:[email protected]>; Joyce, Michael J >(398J); rcmes-dev; >[email protected] <mailto:[email protected]> >Subject: Re: failed to run rcmes following the step-by-step document > > >Hi Kim, > >Thank you for your help. I'm sorry but I'm getting confused with all >these different packages I received from you and your colleagues!. The >last one you just sent to me still contains params = db.getParams() in >rcmet.py at line 194. I've replaced this file > with the one sent by Cameron before and it seems to be ok. But when I've >tried to reproduce the result you sent to me (with obsSource=1) I got >this error : > >... >data_prep is completed: both obs and mdl data are re-gridded to a common >analysis grid >Traceback (most recent call last): > File "rcmet.py", line 299, in <module> > runUsingConfig(args.CONFIG) > File "rcmet.py", line 237, in runUsingConfig > >(jobProperties,obsSource,obsDatasetList,obsList,obsVarName,obsLonName,obsL >atName,obsTimeName,obsTimestep,gridBox,models) >ValueError: too many values to unpack > >Do you know how can I fix this please? > >NOTE: with obsSource=0 I got : >No input observation data file. EXIT > >Thanks and regards >Rashyd > > > >2014-05-29 20:08 GMT+04:00 Kim, Jinwon ><[email protected]<mailto:[email protected]>>: >To all, > >*** Please update your RCMET with the attached version 2.1.3. *** > >I found that some files included in the earlier tar file for the release >version 2.1.3 have some critical issues. Now I've applied all fixes and >successfully ran a test case with a MENA domain dataset; the version >works properly now. I apologize for inconvenience. > >Rashyd: >Please see the two attached figures for the MENA domain (model bias, GPCP >climatology for 1991-2000 over the MENA domain) for your reference. The >configure file for this processing is: >"rcmet2.1.3/src/main/python/rcmes/cordexcordexMENA.cfg". The analysis >domain has been adjusted to fit the MENA domain. > > >Aanjay and Rama: >The code also works fine with the India domain. The two attached figures >for the India domain shows the TRMM climatology for 98-2000 and the bias >of model ensemble against TRMM for the same period. The sample configure >file is: cordexIndia.cfg under resources/ > (the same location as the MENA config file). > >Please contact me/mike/rcmes-dev if you have any questions. > > >With best regards, >----------------------------------------------------------------- >Jinwon Kim >Joint Institute for Regional Earth System Science and Engineering and >Dept. Atmospheric and Oceanic Sciences >University of California, Los Angeles >607 Charles E Young Drive East, Young Hall, Room 4242, Los Angeles, CA >90095-7228, USA > >[email protected]<mailto:[email protected]>; 310-206-2828 >________________________________________ >From: Rashyd ZAABOUL [[email protected]<mailto:[email protected]>] > >Sent: Thursday, May 29, 2014 12:07 AM >To: Kim, Jinwon >Cc: Joyce, Michael J (398J); rcmes-dev >Subject: Re: failed to run rcmes following the step-by-step document > >Hi Michael, >The package you've just sent to me contains the same rcmet.py as the >package 2.1.2 and has on line 194 : > params = db.getParams() > >Is it the right one? Because I'm still having the same error : > >Running using config file: ../rcmes/resources/cordexAF.cfg >Traceback (most recent call last): > File "rcmet.py", line 299, in <module> > runUsingConfig(args.CONFIG) > File "rcmet.py", line 194, in runUsingConfig > params = db.getParams() >AttributeError: 'module' object has no attribute 'getParams' > >I've attached one of the files that I'm trying to use. > >Thanks again >Rashyd > > > > >2014-05-29 10:29 GMT+04:00 Kim, Jinwon ><[email protected]<mailto:[email protected]><mailto:[email protected] >u<mailto:[email protected]>>>: >Here comes the attachment. > >----------------------------------------------------------------- >Jinwon Kim >Joint Institute for Regional Earth System Science and Engineering and >Dept. Atmospheric and Oceanic Sciences >University of California, Los Angeles >607 Charles E Young Drive East, Young Hall, Room 4242, Los Angeles, CA >90095-7228, USA > >[email protected]<mailto:[email protected]><mailto:[email protected] ><mailto:[email protected]>>; > 310-206-2828 >________________________________________ >From: Rashyd ZAABOUL >[[email protected]<mailto:[email protected]><mailto:[email protected]<mail >to:[email protected]>>] > >Sent: Wednesday, May 28, 2014 10:27 PM >To: Joyce, Michael J (398J) >Cc: Kim, Jinwon; rcmes-dev >Subject: Re: failed to run rcmes following the step-by-step document > >Hi Michael, > >With the new Jinwon's RCMET version 2.1.3 I got this error : > >Running using config file: ../rcmes/resources/cordexAF.cfg >Traceback (most recent call last): > File "rcmet.py", line 299, in <module> > runUsingConfig(args.CONFIG) > File "rcmet.py", line 194, in runUsingConfig > params = db.getParams() >AttributeError: 'module' object has no attribute 'getParams' > >My >PYTHONPATH=/home/wrf/rcmet-2.1.3:/home/wrf/rcmet-2.1.3/src/main/python/rcm >es > >I'm using files for precipitation exactly as I download them from ESGF >portal for CORDEX MENA domain. The files I'm using are : > >pr_MNA-44_CNRM-CERFACS-CNRM-CM5_historical_r1i1p1_SMHI-RCA4_v1_mon_197101- >198012.nc >pr_MNA-44_CNRM-CERFACS-CNRM-CM5_historical_r1i1p1_SMHI-RCA4_v1_mon_198101- >199012.nc >pr_MNA-44_CNRM-CERFACS-CNRM-CM5_historical_r1i1p1_SMHI-RCA4_v1_mon_199101- >200012.nc >pr_MNA-44_CNRM-CERFACS-CNRM-CM5_historical_r1i1p1_SMHI-RCA4_v1_mon_200101- >200512.nc >pr_MNA-44_ICHEC-EC-EARTH_historical_r12i1p1_SMHI-RCA4_v1_mon_197101-198012 >.nc >pr_MNA-44_ICHEC-EC-EARTH_historical_r12i1p1_SMHI-RCA4_v1_mon_198101-199012 >.nc >pr_MNA-44_ICHEC-EC-EARTH_historical_r12i1p1_SMHI-RCA4_v1_mon_199101-200012 >.nc >pr_MNA-44_ICHEC-EC-EARTH_historical_r12i1p1_SMHI-RCA4_v1_mon_200101-200512 >.nc >pr_MNA-44_NOAA-GFDL-GFDL-ESM2M_historical_r1i1p1_SMHI-RCA4_v1_mon_197101-1 >98012.nc >pr_MNA-44_NOAA-GFDL-GFDL-ESM2M_historical_r1i1p1_SMHI-RCA4_v1_mon_198101-1 >99012.nc >pr_MNA-44_NOAA-GFDL-GFDL-ESM2M_historical_r1i1p1_SMHI-RCA4_v1_mon_199101-2 >00012.nc >pr_MNA-44_NOAA-GFDL-GFDL-ESM2M_historical_r1i1p1_SMHI-RCA4_v1_mon_200101-2 >00512.nc > > >Do I have to pre-process this data first before I can use it with RCMET ? > >I've attached my config file to check if I'm missing something. > >Thanks for your help > >Rashyd > > > > >2014-05-28 18:05 GMT+04:00 Joyce, Michael J (398J) ><[email protected]<mailto:[email protected]><mailto: >[email protected]<mailto:[email protected]>><mailto: >[email protected]<mailto:[email protected]><mailto:M >[email protected]<mailto:[email protected]>>>>: >Hi Rashyd, > >It looks like your¹e using one of Jinwon¹s RCMET versions. He recently >released a newer version that may resolve some of your problems. You can >find and download it at [1]. I¹m sure Jinwon will be able to help you get >this resolved. Thank you for sending the > document you used. I¹ll be sure to get it and the website updated with >the correct information. > >[1] >https://github.com/apache/climate/tree/jinwon-rcmet-2.1.3 ><https://github.com/apache/climate/tree/jinwon-rcmet-2.1.3> > >Thanks much! > >-- >Michael J. Joyce >Engineering Applications Software Engineer >Instrument Software and Science Data Systems >NASA Jet Propulsion Laboratory >California Institute of Technology >4800 Oak Grove Drive >Pasadena, California 91109 >Mail Stop: 158-242 >Tel: (818) 354-7550 >Fax: (818) 393-1370 > > > >From: Rashyd ZAABOUL ><[email protected]<mailto:[email protected]><mailto:[email protected]<mail >to:[email protected]>><mailto:[email protected]<mailto:[email protected]>< >mailto:[email protected]<mailto:[email protected]>>>> >Date: Wednesday, May 28, 2014 at 5:09 AM > >To: "Kim, Jinwon (329M-Affiliate)" ><[email protected]<mailto:[email protected]><mailto:[email protected] >u<mailto:[email protected]>><mailto:[email protected]<mailto:jkim@atmo >s.ucla.edu><mailto:[email protected]<mailto:[email protected]>>>> >Cc: rcmes-dev ><[email protected]<mailto:[email protected]><mailto:rcmes-dev@jp >l.nasa.gov<mailto:[email protected]>><mailto:[email protected]<m >ailto:[email protected]><mailto:[email protected]<mailto:rcmes-d >[email protected]>>>>, > JPL ><[email protected]<mailto:[email protected]><mailto: >[email protected]<mailto:[email protected]>><mailto: >[email protected]<mailto:[email protected]><mailto:M >[email protected]<mailto:[email protected]>>>> >Subject: Re: failed to run rcmes following the step-by-step document > >Hi Jim and Michael, > >I've modified the script >~rcmet-2.1.2/src/main/python/rcmes/storage/rcmed.py by replacing >json.loads function by simplejson.loads one to get more details on the >error and this is what I got : > >Running using config file: ../rcmes/resources/mena.cfg >Traceback (most recent call last): > File "rcmet.py", line 299, in <module> > runUsingConfig(args.CONFIG) > File "rcmet.py", line 194, in runUsingConfig > params = db.getParams() > File "/home/rcmes/rcmet-2.1.2/src/main/python/rcmes/storage/rcmed.py", >line 27, in getParams > paramDict = simplejson.loads(httpRequest.text) > File >"/home/rcmes/anaconda/lib/python2.7/site-packages/simplejson-3.5.2-py2.7-l >inux-x86_64.egg/simplejson/__init__.py", line 501, in loads > return _default_decoder.decode(s) > File >"/home/rcmes/anaconda/lib/python2.7/site-packages/simplejson-3.5.2-py2.7-l >inux-x86_64.egg/simplejson/decoder.py", line 370, in decode > obj, end = self.raw_decode(s) > File >"/home/rcmes/anaconda/lib/python2.7/site-packages/simplejson-3.5.2-py2.7-l >inux-x86_64.egg/simplejson/decoder.py", line 389, in raw_decode > return self.scan_once(s, idx=_w(s, idx).end()) >simplejson.scanner.JSONDecodeError: Expecting value: line 1 column 1 >(char 0) > >What does this error mean ? I'm afraid the database RCMED is down while >I'm trying to get data out of it ! > >Best regards >Rashyd > > > > >2014-05-27 8:13 GMT-07:00 Kim, Jinwon ><[email protected]<mailto:[email protected]><mailto:[email protected] >u<mailto:[email protected]>><mailto:[email protected]<mailto:jkim@atmo >s.ucla.edu><mailto:[email protected]<mailto:[email protected]>>>>: >Thanks for your interests. > >For organizing your data: > >You can put your model data in your own directory. Then specify the link >to the data (or data files) in the configure file, [MODEL] block. The >code currently takes netcdf format files for input. > >If you use the observed data in the RCMES database, just specify the >parameter numbers in the [RCMED] block, also in the configure file. If >you use your own observation data, you can specify it in the same block >(it also takes netcdf-format files). > >Hope this helps. > > >With best regards, > >----------------------------------------------------------------- >Jinwon Kim >Joint Institute for Regional Earth System Science and Engineering and >Dept. Atmospheric and Oceanic Sciences >University of California, Los Angeles >607 Charles E Young Drive East, Young Hall, Room 4242, Los Angeles, CA >90095-7228, USA > >[email protected]<mailto:[email protected]><mailto:[email protected] ><mailto:[email protected]>><mailto:[email protected]<mailto:jkim@atmos >.ucla.edu><mailto:[email protected]<mailto:[email protected]>>>; > 310-206-2828 >________________________________________ >From: Rashyd ZAABOUL >[[email protected]<mailto:[email protected]><mailto:[email protected]<mail >to:[email protected]>><mailto:[email protected]<mailto:[email protected]>< >mailto:[email protected]<mailto:[email protected]>>>] >Sent: Tuesday, May 27, 2014 7:59 AM > >To: >[email protected]<mailto:[email protected]><mailto:rcmes-dev@jpl >.nasa.gov<mailto:[email protected]>><mailto:[email protected]<ma >ilto:[email protected]><mailto:[email protected]<mailto:rcmes-de >[email protected]>>> >Subject: failed to run rcmes following the step-by-step document > >Hi >I'm trying to use RCMES to evaluate CORDEX models outputs against CRU, >TRMM ... I followed the step-by-step document but it seems that something >is missing there. For example it doesn't tell how to organize my data? >where to put the observations ... > >Could you please provide me with any useful documentation that can help >me setup RCMES to evaluate those models? > >Thanks >Rashyd > >-- >Rashyd Zaaboul >Climate and Water Modeling Scientist >International Center for Biosaline Agriculture >P.O. Box 14660 Dubai, UAE >Mobile : +971 50 675 2007 >Office : +971 4 336 1100 x 259 > >[email protected]<mailto:[email protected]><mailto:r.zaa >[email protected]<mailto:[email protected]>><mailto:r.zaaboul >@biosaline.org.ae<mailto:[email protected]><mailto:r.zaaboul@bios >aline.org.ae<mailto:[email protected]>>><mailto:r.zaaboul@biosali >ne.org.ae<mailto:[email protected]><mailto:[email protected] >g.ae<mailto:[email protected]>><mailto:[email protected] ><mailto:[email protected]><mailto:[email protected]<mail >to:[email protected]>>>> > >Please consider the environment before printing this e-mail > > > >-- >Rashyd Zaaboul >Climate and Water Modeling Scientist >International Center for Biosaline Agriculture >P.O. Box 14660 Dubai, UAE >Mobile : +971 50 675 2007 >Office : +971 4 336 1100 x 259 >[email protected]<mailto:[email protected]><mailto:r.zaa >[email protected]<mailto:[email protected]>><mailto:r.zaaboul >@biosaline.org.ae<mailto:[email protected]><mailto:r.zaaboul@bios >aline.org.ae<mailto:[email protected]>>> > >Please consider the environment before printing this e-mail > > > >-- >Rashyd Zaaboul >Climate and Water Modeling Scientist >International Center for Biosaline Agriculture >P.O. Box 14660 Dubai, UAE >Mobile : +971 50 675 2007 >Office : +971 4 336 1100 x 259 >[email protected]<mailto:[email protected]><mailto:r.zaa >[email protected]<mailto:[email protected]>><mailto:r.zaaboul >@biosaline.org.ae<mailto:[email protected]><mailto:r.zaaboul@bios >aline.org.ae<mailto:[email protected]>>> > >Please consider the environment before printing this e-mail > > > >-- >Rashyd Zaaboul >Climate and Water Modeling Scientist >International Center for Biosaline Agriculture >P.O. Box 14660 Dubai, UAE >Mobile : +971 50 675 2007 >Office : +971 4 336 1100 x 259 >[email protected]<mailto:[email protected]><mailto:r.zaa >[email protected]<mailto:[email protected]>> > >Please consider the environment before printing this e-mail > > > >-- >Rashyd Zaaboul >Climate and Water Modeling Scientist >International Center for Biosaline Agriculture >P.O. Box 14660 Dubai, UAE >Mobile : +971 50 675 2007 >Office : +971 4 336 1100 x 259 >[email protected]<mailto:[email protected]> > >Please consider the environment before printing this e-mail > > > > > > > > >-- >Rashyd Zaaboul > >Climate and Water Modeling Scientist > >International Center for Biosaline Agriculture >P.O. Box 14660 Dubai, UAE >Mobile : +971 50 675 2007 >Office : +971 4 336 1100 x 259 > >[email protected] > >Please consider the environment before printing this e-mail > >
