On 5/23/14 12:04 AM, Hedonil wrote:
Another effect since ~today.
Output of longer lists get truncated in the middle of output.
/var filled up on the tools web proxy, which was causing the proxy to
misbehave in a variety of ways, mainly because nginx was unable to write
to /var/log/nginx/access.log and flipped out as a result.
I applied the role::labs::lvm::biglogs class to the proxy instance,
which designates 8g of space for logging, then rebooted the box, and
then scrambled a bit to recreate all of the /var/log/<whatever> dirs
that services expected to find but were too stupid to create. After
that the proxy service appears to have resumed normal behavior.
8g is a fair bit of space, and the nginx logs are already set to rotate
out every 2-3 days, so there /should/ be plenty of room now.
Nevertheless, there's some chance that the log overflow problem was a
secondary symptom of a different problem that caused extra logging in
the first place, so I'm going to investigate that a bit more and also
keep an eye out.
Sorry for the service interruption!
-Andrew
_______________________________________________
Labs-l mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/labs-l
