Oscar, I think “pip install msnoise -U” should do the trick (-U is for —upgrade).
Lukas 2017-06-02 18:55 GMT+02:00 Oscar Alberto Castro Artola <oscar.cas....@gmail.com>: > Dear Christos and Lukas, > > first of all thanks for you help! I really apreciate it! > > Christos: I have a server with 40 cores, so I think the *-t 20* flag is > working fine! > > Lukas: I tried to do a *pip install msnoise*, but I don't think it is > actually upgrading MSNoise, the output is: > > *Requirement already satisfied: msnoise in > /usr/local/anaconda/lib/python2.7/site-packages* > *Requirement already satisfied: setuptools in > /usr/local/anaconda/lib/python2.7/site-packages (from msnoise)* > *Requirement already satisfied: numpy>=1.0 in > /usr/local/anaconda/lib/python2.7/site-packages (from msnoise)* > *Requirement already satisfied: scipy in > /usr/local/anaconda/lib/python2.7/site-packages (from msnoise)* > *Requirement already satisfied: pandas in > /usr/local/anaconda/lib/python2.7/site-packages (from msnoise)* > *Requirement already satisfied: matplotlib in > /usr/local/anaconda/lib/python2.7/site-packages (from msnoise)* > *Requirement already satisfied: statsmodels in > /usr/local/anaconda/lib/python2.7/site-packages (from msnoise)* > *Requirement already satisfied: sqlalchemy in > /usr/local/anaconda/lib/python2.7/site-packages (from msnoise)* > *Requirement already satisfied: obspy in > /usr/local/anaconda/lib/python2.7/site-packages (from msnoise)* > *Requirement already satisfied: click in > /usr/local/anaconda/lib/python2.7/site-packages/click-4.0-py2.7.egg (from > msnoise)* > *Requirement already satisfied: pymysql in > /usr/local/anaconda/lib/python2.7/site-packages (from msnoise)* > *Requirement already satisfied: flask in > /usr/local/anaconda/lib/python2.7/site-packages (from msnoise)* > *Requirement already satisfied: flask-admin in > /usr/local/anaconda/lib/python2.7/site-packages (from msnoise)* > *Requirement already satisfied: multiprocessing_logging in > /usr/local/anaconda/lib/python2.7/site-packages (from msnoise)* > *Requirement already satisfied: markdown in > /usr/local/anaconda/lib/python2.7/site-packages (from msnoise)* > *Requirement already satisfied: folium in > /usr/local/anaconda/lib/python2.7/site-packages (from msnoise)* > *Requirement already satisfied: six>=1.10 in > /usr/local/anaconda/lib/python2.7/site-packages (from matplotlib->msnoise)* > *Requirement already satisfied: python-dateutil in > /usr/local/anaconda/lib/python2.7/site-packages (from matplotlib->msnoise)* > *Requirement already satisfied: functools32 in > /usr/local/anaconda/lib/python2.7/site-packages (from matplotlib->msnoise)* > *Requirement already satisfied: subprocess32 in > /usr/local/anaconda/lib/python2.7/site-packages (from matplotlib->msnoise)* > *Requirement already satisfied: pytz in > /usr/local/anaconda/lib/python2.7/site-packages (from matplotlib->msnoise)* > *Requirement already satisfied: cycler>=0.10 in > /usr/local/anaconda/lib/python2.7/site-packages/cycler-0.10.0-py2.7.egg > (from matplotlib->msnoise)* > *Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=1.5.6 in > /usr/local/anaconda/lib/python2.7/site-packages (from matplotlib->msnoise)* > *Requirement already satisfied: future>=0.12.4 in > /usr/local/anaconda/lib/python2.7/site-packages (from obspy->msnoise)* > *Requirement already satisfied: lxml in > /usr/local/anaconda/lib/python2.7/site-packages (from obspy->msnoise)* > *Requirement already satisfied: decorator in > /usr/local/anaconda/lib/python2.7/site-packages (from obspy->msnoise)* > *Requirement already satisfied: requests in > /usr/local/anaconda/lib/python2.7/site-packages (from obspy->msnoise)* > *Requirement already satisfied: wtforms in > /usr/local/anaconda/lib/python2.7/site-packages (from flask-admin->msnoise)* > *Requirement already satisfied: Jinja2 in > /usr/local/anaconda/lib/python2.7/site-packages (from folium->msnoise)* > > This is the second time I do *msnoise -t 20 compute_mwcs*, last time it > worked fine but I needed to change *dtt_lag* to "dynamic". So when I tried > to rerun* msnoise -t 20 compute_mwcs* with the new parameter this error > message appears. What I've done is run the process in the morning, all day > it works fine, but the next morning I always find that error message. > > Thanks again! > > Oscar > > On Fri, Jun 2, 2017 at 8:39 AM, Lukas Preiswerk <preisw...@vaw.baug.ethz.ch> > wrote: > >> Hi Oscar >> >> The problem is that “MySQL server has gone away”, so it can’t update >> the database after finishing the MWCS job. We fixed this in the newest >> version, so you should indeed update. >> >> I think that updating is done by “pip install msnoise”, and then for >> each project “msnoise upgrade_db” (see the release notes >> http://msnoise.org/doc/releasenotes/msnoise-1.5.html ). Tom can >> clarify if this method I just described is actually the way to go… >> >> Best, Lukas >> >> >> >> 2017-06-02 15:20 GMT+02:00 Christos Evangelidis <ce...@noa.gr>: >> > Dear Oscar, >> > >> > Can you really run 20 instances of the code (-t 20) ? I find the -t 20 >> > rather high, but your machine might support it.. >> > >> > Regards, >> > >> > Christos >> > >> > Στις 1 Ιουν 2017 6:01 μμ, ο χρήστης "Oscar Alberto Castro Artola" < >> > oscar.cas....@gmail.com> έγραψε: >> > >> > Hello, >> > >> > I have 10 years of data for 13 stations, I have computed the CCF and now >> it >> > is turn to compute the MWCS. When I do *msnoise -t 20 compute_mwcs *this >> > error appears: >> > >> > *Process Process-4:* >> > *Traceback (most recent call last):* >> > * File "/usr/local/anaconda/lib/python2.7/multiprocessing/process.py", >> > line 258, in _bootstrap* >> > * self.run()* >> > * File "/usr/local/anaconda/lib/python2.7/multiprocessing/process.py", >> > line 114, in run* >> > * self._target(*self._args, **self._kwargs)* >> > * File >> > "/usr/local/anaconda/lib/python2.7/site-packages/msnoise/ >> > s05compute_mwcs.py", >> > line 150, in main* >> > * update_job(db, day, pair, jobtype='DTT', flag='D')* >> > * File "/usr/local/anaconda/lib/python2.7/site-packages/ >> msnoise/api.py", >> > line 615, in update_job* >> > * job = session.query(Job).filter(Job.day == day).filter(Job.pair == >> > pair).filter(Job.jobtype == jobtype).first()* >> > * File >> > "/usr/local/anaconda/lib/python2.7/site-packages/ >> sqlalchemy/orm/query.py", >> > line 2755, in first* >> > * ret = list(self[0:1])* >> > * File >> > "/usr/local/anaconda/lib/python2.7/site-packages/ >> sqlalchemy/orm/query.py", >> > line 2547, in __getitem__* >> > * return list(res)* >> > * File >> > "/usr/local/anaconda/lib/python2.7/site-packages/ >> sqlalchemy/orm/query.py", >> > line 2855, in __iter__* >> > * return self._execute_and_instances(context)* >> > * File >> > "/usr/local/anaconda/lib/python2.7/site-packages/ >> sqlalchemy/orm/query.py", >> > line 2878, in _execute_and_instances* >> > * result = conn.execute(querycontext.statement, self._params)* >> > * File >> > "/usr/local/anaconda/lib/python2.7/site-packages/ >> sqlalchemy/engine/base.py", >> > line 945, in execute* >> > * return meth(self, multiparams, params)* >> > * File >> > "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/ >> > sql/elements.py", >> > line 263, in _execute_on_connection* >> > * return connection._execute_clauseelement(self, multiparams, >> params)* >> > * File >> > "/usr/local/anaconda/lib/python2.7/site-packages/ >> sqlalchemy/engine/base.py", >> > line 1053, in _execute_clauseelement* >> > * compiled_sql, distilled_params* >> > * File >> > "/usr/local/anaconda/lib/python2.7/site-packages/ >> sqlalchemy/engine/base.py", >> > line 1189, in _execute_context* >> > * context)* >> > * File >> > "/usr/local/anaconda/lib/python2.7/site-packages/ >> sqlalchemy/engine/base.py", >> > line 1402, in _handle_dbapi_exception* >> > * exc_info* >> > * File >> > "/usr/local/anaconda/lib/python2.7/site-packages/ >> sqlalchemy/util/compat.py", >> > line 203, in raise_from_cause* >> > * reraise(type(exception), exception, tb=exc_tb, cause=cause)* >> > * File >> > "/usr/local/anaconda/lib/python2.7/site-packages/ >> sqlalchemy/engine/base.py", >> > line 1182, in _execute_context* >> > * context)* >> > * File >> > "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/ >> > engine/default.py", >> > line 470, in do_execute* >> > * cursor.execute(statement, parameters)* >> > * File >> > "/usr/local/anaconda/lib/python2.7/site-packages/pymysql/cursors.py", >> line >> > 158, in execute* >> > * result = self._query(query)* >> > * File >> > "/usr/local/anaconda/lib/python2.7/site-packages/pymysql/cursors.py", >> line >> > 308, in _query* >> > * conn.query(q)* >> > * File >> > "/usr/local/anaconda/lib/python2.7/site-packages/ >> pymysql/connections.py", >> > line 819, in query* >> > * self._execute_command(COMMAND.COM_QUERY, sql)* >> > * File >> > "/usr/local/anaconda/lib/python2.7/site-packages/ >> pymysql/connections.py", >> > line 1037, in _execute_command* >> > * self._write_bytes(packet)* >> > * File >> > "/usr/local/anaconda/lib/python2.7/site-packages/ >> pymysql/connections.py", >> > line 989, in _write_bytes* >> > * raise err.OperationalError(2006, "MySQL server has gone away (%r)" % >> > (e,))* >> > *OperationalError: (pymysql.err.OperationalError) (2006, "MySQL server >> has >> > gone away (error(32, 'Broken pipe'))") [SQL: u'SELECT jobs.ref AS >> jobs_ref, >> > jobs.day AS jobs_day, jobs.pair AS jobs_pair, jobs.jobtype AS >> jobs_jobtype, >> > jobs.flag AS jobs_flag, jobs.lastmod AS jobs_lastmod \nFROM jobs \nWHERE >> > jobs.day = %(day_1)s AND jobs.pair = %(pair_1)s AND jobs.jobtype = >> > %(jobtype_1)s \n LIMIT %(param_1)s'] [parameters: {u'param_1': 1, >> u'day_1': >> > '2009-02-16', u'jobtype_1': 'DTT', u'pair_1': 'IG.ARIG:IG.MOIG'}]* >> > >> > At the beginning everything works fine, but because it takes a long time >> to >> > make the computation (around 5 days), every time that I come back to see >> > the results this happens. I am working with MSNoise 1.4 and I don't know >> > how to upgrade to MSNoise 1.5. >> > >> > Does anyone know how to fix this? >> > >> > Thanks in advance! >> > >> > Oscar >> > _______________________________________________ >> > MSNoise mailing list >> > MSNoise@mailman-as.oma.be >> > http://mailman-as.oma.be/mailman/listinfo/msnoise >> > _______________________________________________ >> > MSNoise mailing list >> > MSNoise@mailman-as.oma.be >> > http://mailman-as.oma.be/mailman/listinfo/msnoise >> _______________________________________________ >> MSNoise mailing list >> MSNoise@mailman-as.oma.be >> http://mailman-as.oma.be/mailman/listinfo/msnoise >> > _______________________________________________ > MSNoise mailing list > MSNoise@mailman-as.oma.be > http://mailman-as.oma.be/mailman/listinfo/msnoise _______________________________________________ MSNoise mailing list MSNoise@mailman-as.oma.be http://mailman-as.oma.be/mailman/listinfo/msnoise
