Re: [Sugar-devel] git.sugarlabs.org down for unplanned maintenance

2014-04-14 Thread Bernie Innocenti
On 04/12/2014 02:07 AM, Sebastian Silva wrote: Here I just got home. Sorry for the inconvenience I might have caused. Bernie, do you know which log was/is growing out of hand? Both access.log and node.sugarlabs.org.log. I discarded the first and compressed the second (it compresses very

Re: [Sugar-devel] git.sugarlabs.org down for unplanned maintenance

2014-04-12 Thread Sebastian Silva
Here I just got home. Sorry for the inconvenience I might have caused. Bernie, do you know which log was/is growing out of hand? Here's a report on everything I know about the issue. We've been experiencing some performance degradation and also some downtime in Sugar Network services (this is

Re: [Sugar-devel] git.sugarlabs.org down for unplanned maintenance

2014-04-12 Thread Gonzalo Odiard
I know nothing about our infraestructure but, is possible run proyects in development, like Sugar Network in a different server/vm than critical services like git? Gonzalo On Sat, Apr 12, 2014 at 6:07 AM, Sebastian Silva sebast...@fuentelibre.orgwrote: Here I just got home. Sorry for the

Re: [Sugar-devel] git.sugarlabs.org down for unplanned maintenance

2014-04-12 Thread Sebastian Silva
If it is necessary it could be moved. Sugar Network at this point is in production and we do have a process for deployment (there is testing and devel instance). Alsroot has had a pretty good track record of keeping git up and running. I think it's a shame people are moving to github. What we

[Sugar-devel] git.sugarlabs.org down for unplanned maintenance

2014-04-11 Thread Bernie Innocenti
I was notified that git.sugarlabs.org was showing errors. After some head scraping I realized that the root filesystem on jita was full. I looked around and found giant request logs containing millions of requests apparently originating from XOs located in Peru. We've been DDOSed by our own

Re: [Sugar-devel] git.sugarlabs.org down for unplanned maintenance

2014-04-11 Thread Gonzalo Odiard
Thanks Berni :) On Sat, Apr 12, 2014 at 12:51 AM, Bernie Innocenti ber...@sugarlabs.orgwrote: I was notified that git.sugarlabs.org was showing errors. After some head scraping I realized that the root filesystem on jita was full. I looked around and found giant request logs containing