On Sun, Mar 27, 2016 at 03:25:40AM -0000, DD Park wrote:
> Hello, I need your help. This bug seemed to have been placed offline due to
> inactivity. It is still a problem as been working on moving things around
> to get a testing platform. I've been getting new hardware, and started
> another build process to get me to a point of testing. I'm plan on doing a
> little more testing before going into production based on the thought that
> this problem was fixed, but initial testing shows I'm stilll having some
> similar problems. I've built a 18TB file system raid5 ext4, and I was
> crossing my fingers that it would be stable, but I'm seeing all kinds of
> corruptions and doing fsck early I see that the file system doesn't stay
> clean for long. I've built 3 systems so far. Two of them have gone into
> production and I've limited my ext4 to 16TB. I built another system with
> 18TB and I once I start copying large amounts of files onto the system, I
> start seeing some warning messages indicating some forms of corruption, and
> I stop the copy, run fsck, and I find I do not have a clean file system.
> I'm running ubuntu 14.04.04 LTS on this test system. I've got another near
> identical setup with ubuntu-14.04 and 16TB or less and works fine(this was
> the original system that I saw my corruption. After downsizing, I'm good).
> I've got another with 18TB, but split into a 16TB partition and 2TB
> partition, on a ubuntu-15.04 system and that is working fine. I go back to
> an hybrid system I built to do this test. It is running ubuntu14.04.04 and
> built this one with 18TB. This was an older file server that did not have
> problems that I decomissioned recently so I could do this testing. I
> started my burn in tests and started seeing corruption of the file system.
> As expected the only thing I can determine is that it doesn't seem to like
> >16TB. Please let me know how I can help get this debugged.
So the original problem was about fsck crashing with a seg fault.
That's different from it finding corruptions. So the first question
is what exactly are you seeing? Corruptions? Fsck crashing? Both?
The next question is are you using resize2fs or not? There are known
problems with using resize2fs with large partitions, especially if you
aren't using the very latest version of e2fsprogs. In general on-line
resizing is going to be much safer than off-line resizing (the bugs
were in resize2fs's off-line resizing code).
If you are seeing it crash, the best thing to do is to get the very
latest version of e2fsprogs, and build it, and then run it from there,
so we can get a stack trace with line numbers. Since I'm about to
release 1.43, ideally you would do this with both 1.42.13 as well as
the tip of the e2fsprogs git's "master" branch.
(Sorry, I don't provide support for distro versions of the kernel and
e2fsprogs. If you want that, you need to pay $$$ to Canonical and get
their enterprise support product offerring.)
- Ted
P.S. Also, to be clear, you are are using software raid?
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to e2fsprogs in Ubuntu.
https://bugs.launchpad.net/bugs/1345682
Title:
fsck on 24TB ext4 keeps crashing
Status in e2fsprogs package in Ubuntu:
Expired
Bug description:
date; fsck -vy /dev/mapper/raid61p1 ;date
Thu Jul 17 11:54:21 PDT 2014
fsck from util-linux 2.20.1
I can't get my ext4 24TB to fsck clean. there came a whole bunch of file stat
problems after a clean reboot.
I tried 3 times so far, and each does something like segfault. I looked
twice, and the errors were different,
but was able to capture one output. The system has 6GB ram, and it used up
all the memory. ended up adding 25GB
of swap as fsck seems to use up a huge amount of memory. After a long while,
usually after mem use is >90%,
I come back to see the fsck has crashed and file system is still not clean.
The reboot was clean and should not have caused any corruption. the
system is using ubuntu-14.04
# uname -a
Linux gigabyte133 3.13.0-32-generic #57-Ubuntu SMP Tue Jul 15 03:51:08 UTC
2014 x86_64 x86_64 x86_64 GNU/Linux
root@gigabyte133:~#
I had to use the 64 bit version since the 32 bit version I couldn't get more
than 15TB file system built and read
that there were compatibility problems between 32bit version with the -O
64bit and the 64 bit version so just
built the whole thing in 64 bit ubuntu..
e2fsck 1.42.9 (4-Feb-2014)
/dev/mapper/raid61p1 contains a file system with errors, check forced.
Pass 1: Checking inodes, blocks, and sizes
Inode 203167820 has compression flag set on filesystem without compression
support. Clear? yes
Signal (11) SIGSEGV si_code=SI_KERNEL fault addr=(nil)
fsck.ext4[0x4266f1]
/lib/x86_64-linux-gnu/libc.so.6(+0x36ff0)[0x7fd38d087ff0]
/lib/x86_64-linux-gnu/libext2fs.so.2(ext2fs_rb_next+0x23)[0x7fd38dc7bc43]
/lib/x86_64-linux-gnu/libext2fs.so.2(+0x10670)[0x7fd38dc5e670]
fsck.ext4[0x4100f6]# lsb_release -va
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04 LTS
Release: 14.04
Codename: trusty
/lib/x86_64-linux-gnu/libext2fs.so.2(+0x11c3d)[0x7fd38dc5fc3d]
/lib/x86_64-linux-gnu/libext2fs.so.2(+0x11f78)[0x7fd38dc5ff78]
/lib/x86_64-linux-gnu/libext2fs.so.2(ext2fs_block_iterate3+0xa13)[0x7fd38dc60b13]
fsck.ext4[0x4115c4]
fsck.ext4[0x412699]
fsck.ext4[0x412761]
/lib/x86_64-linux-gnu/libext2fs.so.2(ext2fs_get_next_inode_full+0x59)[0x7fd38dc6c9a9]
fsck.ext4(e2fsck_pass1+0x8d8)[0x4130c8]
fsck.ext4(e2fsck_run+0x52)[0x40deb2]
fsck.ext4(main+0xd27)[0x40a0e7]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x7fd38d072ec5]
fsck.ext4[0x40bde6]
Thu Jul 17 15:55:01 PDT 2014
# lsb_release -va
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04 LTS
Release: 14.04
Codename: trusty
ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: util-linux 2.20.1-5.1ubuntu20.1
ProcVersionSignature: Ubuntu 3.13.0-32.57-generic 3.13.11.4
Uname: Linux 3.13.0-32-generic x86_64
NonfreeKernelModules: zfs zunicode zavl zcommon znvpair
ApportVersion: 2.14.1-0ubuntu3.2
Architecture: amd64
Date: Sat Jul 19 18:49:32 2014
InstallationDate: Installed on 2014-07-11 (8 days ago)
InstallationMedia: Ubuntu 14.04 LTS "Trusty Tahr" - Release amd64 (20140417)
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
SourcePackage: util-linux
UpgradeStatus: No upgrade log present (probably fresh install)
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/e2fsprogs/+bug/1345682/+subscriptions
--
Mailing list: https://launchpad.net/~touch-packages
Post to : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help : https://help.launchpad.net/ListHelp
