On Saturday 03 of November 2018 13:56:05 Slávek Banko wrote: > On Wednesday 31 of October 2018 22:23:22 Michal Čihař wrote: > > Hello > > > > On Sat, 2018-10-27 at 12:17 +0200, Slávek Banko wrote: > > > I'm trying to connect Gitea and Weblate. Because in Gitea I did not > > > see > > > the option to setting up a web-hook with Authorization sending in > > > the > > > header, I the used deprecated way > > > "/hooks/update/<project>/<component>". > > > > These are deprecated, but will most likely not go away in near > > future. > > > > Anyway best would be to have native support for Gitea, adding it > > should not be hard, here you can see how I did that for Pagure: > > > > https://github.com/WeblateOrg/weblate/pull/2331 > > > > > In Gitea I see that the web-hook was successfully sent, from > > > Weblate I see > > > the answer: {"status": "success", "message": "Update triggered"}. > > > But > > > there is no activity visible in the Weblate. > > > > Check Celery logs if it's processing the background tasks. > > It seems to be the culprit the celery. There is a crash in the worker > that handles scheduled tasks: > > [2018-11-03 12:41:09,496:INFO/ForkPoolWorker-2] Lock 140118536421840 > acquired on /var/lib/weblate/data/vcs/applications/knetload.lock > [2018-11-03 12:41:09,641: INFO/ForkPoolWorker-2] Lock 140118536421840 > released on /var/lib/weblate/data/vcs/applications/knetload.lock > [2018-11-03 12:41:10,716: ERROR/ForkPoolWorker-2] Error: > Exception('This file is already closed',) > Traceback (most recent call last): > > File > "/var/lib/weblate/local/lib/python2.7/site-packages/celery_batches/__in >it__.py", line 148, in apply_batches_task > result = task(*args) > > File > "/var/lib/weblate/local/lib/python2.7/site-packages/celery/app/trace.py >", line 642, in __protected_call__ > return orig(self, *args, **kwargs) > > File > "/var/lib/weblate/local/lib/python2.7/site-packages/celery/app/task.py" >, line 375, in __call__ > return self.run(*args, **kwargs) > > File > "/var/lib/weblate/local/lib/python2.7/site-packages/weblate/trans/searc >h.py", line 261, in update_fulltext > fulltext.update_index(unitdata) > > File > "/var/lib/weblate/local/lib/python2.7/site-packages/weblate/trans/searc >h.py", line 109, in update_index > self.update_source_unit_index(writer, unit) > > File > "/var/lib/weblate/local/lib/python2.7/site-packages/whoosh/writing.py", > line 1181, in __exit__ > self.close() > > File > "/var/lib/weblate/local/lib/python2.7/site-packages/whoosh/writing.py", > line 1217, in close > self.commit(restart=False) > > File > "/var/lib/weblate/local/lib/python2.7/site-packages/whoosh/writing.py", > line 1229, in commit > self.writer.commit(**self.commitargs) > > File > "/var/lib/weblate/local/lib/python2.7/site-packages/whoosh/writing.py", > line 927, in commit > finalsegments.append(self._finalize_segment()) > > File > "/var/lib/weblate/local/lib/python2.7/site-packages/whoosh/writing.py", > line 866, in _finalize_segment > self._flush_segment() > > File > "/var/lib/weblate/local/lib/python2.7/site-packages/whoosh/writing.py", > line 830, in _flush_segment > self.perdocwriter.close() > > File > "/var/lib/weblate/local/lib/python2.7/site-packages/whoosh/codec/whoosh >3.py", line 267, in close > self._cols.save_as_files(self._storage, self._column_filename) > > File > "/var/lib/weblate/local/lib/python2.7/site-packages/whoosh/filedb/compo >und.py", line 297, in save_as_files > for name, blocks in self._readback(): > > File > "/var/lib/weblate/local/lib/python2.7/site-packages/whoosh/filedb/compo >und.py", line 278, in _readback > temp.close() > > File > "/var/lib/weblate/local/lib/python2.7/site-packages/whoosh/filedb/struc >tfile.py", line 123, in close > raise Exception("This file is already closed") > Exception: This file is already closed > > > By the way, I had to set "--concurrency=1", because the individual > worker will quickly load 790 MiB RAM after the startup and the memory > consumption then rises steadily. Now I was restarting the celery when > the consumption was over 1600 MiB RAM. Is this normal behavior? > > Thank you. > > Cheers
Several "rebuild_index --clean" seem to solve the crashes in celery. However, huge memory consumption still remains. Due to the fact that the celeries are gradually processing tasks that have been delayed due to the previous crashes, an update of the repositories was made in connection with the earlier calls of hooks. Cheers -- Slávek
signature.asc
Description: This is a digitally signed message part.
_______________________________________________ Weblate mailing list [email protected] https://lists.cihar.com/cgi-bin/mailman/listinfo/weblate
