On Mon, Feb 5, 2018 at 9:27 AM, Thanh Ha <thanh...@linuxfoundation.org> wrote:
> Hi Everyone, > > We're temporarily taking Nexus down. This is unscheduled maintenance as we > have a serious issue with Nexus not being able to create new files at the > moment. On Saturday Nexus ran out of inodes on the file system and we > temporarily mitigated it while we are waiting for data to be migrated to a > new file system that an address more inodes. Unfortunately we hit this > issue again this morning (which is why many jobs failed with 500 - Internal > Server Error) when they try to upload artifacts to Nexus. > > We're going to be removing some older logs on the server to mitigate this > issue while our data migration is still continuing. Nexus is being shutdown > temporarily so we can restart it after we free up some inodes. > > Regards, > Thanh > We've cleared up about 3 million inodes and restarted Nexus. Jenkins is now back online and should be starting jobs again shortly. We're going to look into removing older logs from the system as logs create many small files which means they are easy targets to get back some inode space... Normally we keep logs for 6 months but we will look into removing 4 month old logs to get back some more inodes while our server continues to migrate data to the new file system. Apologies for the inconvenience. Regards, Thanh
_______________________________________________ infrastructure mailing list infrastructure@lists.opendaylight.org https://lists.opendaylight.org/mailman/listinfo/infrastructure
