How would one go about rebuilding /var?

I have a server running Ubuntu 10.04 with /var on an LVM.  The
original size of /var was about 30 GB, so I tried to expand it by
first expanding the logical volume (lvresize --size 500G) and then
resizing the filesystem (resize2fs).  Expanding the logical volume
went well.  Resizing the filesystem, not so well.  I don't recall the
error message and it has since fallen off the scroll history, but
resize2fs complained of running out of something and couldn't resize.
So, I unmounted /var and ran the requisite e2fsck.  Errors galore.
Here's a small sample:

<quote>
e2fsck 1.41.11 (14-Mar-2010)
e2fsck: Group descriptors look bad... trying backup blocks...
Superblock has an invalid journal (inode 8).
Clear? yes

*** ext3 journal has been deleted - filesystem is now ext2 only ***

Block bitmap for group 0 is not in group.  (block 876163123)
Relocate? yes

Inode table for group 0 is not in group.  (block 0)
WARNING: SEVERE DATA LOSS POSSIBLE.
Relocate? yes

One or more block group descriptor checksums are invalid.  Fix? yes
...
</quote>

After it finished and I remounted it, the only thing on /var was
lost+found.  That means my logs, lockfiles, pids, dpkg database, etc.
are all gone.  The good news is that with a bit of luck I have
recreated enough of /var so that I can ssh into it and become root as
well as restart all of its services.  The other piece of good news is
that this machine was in the initial stages of being customized and
not in production.  The one piece of bad news is that the machine is
at a remote datacenter.  So plopping in a LiveCD is not a viable
option.

How do I go about rebuilding /var?

One option is that I don't.  That is, I move all the services that are
running on this foobared machine to another machine and then rebuild
the foobared machine (which has its own challenges).  But maybe I'm
overlooking something simple.

Regards,
- Robert

-- 
Central West End Linux Users Group (via Google Groups)
Main page: http://www.cwelug.org
To post: [email protected]
To subscribe: [email protected]
To unsubscribe: [email protected]
More options: http://groups.google.com/group/cwelug

Reply via email to