Re: Upgraded from 3.4 to 3.5.1 kernel: machine does not boot
On Sun, Aug 12, 2012 at 10:13 AM, Paul Gortmaker wrote: > On Sun, Aug 12, 2012 at 9:51 AM, Justin Piszcz > wrote: >> On Sun, Aug 12, 2012 at 9:10 AM, Eric Sandeen wrote: >>> On 8/10/12 11:14 PM, Justin Piszcz wrote: On Fri, Aug 10, 2012 at 7:07 PM, Justin Piszcz > > Hi, > > Found the root cause, the 3.5.1 kernel cannot mount my ext4 filesystem > (60TB). >>> >>> You are a brave man running ext4 at 60T, but thank you for testing :) >>> >>> Backing out 8aeb00ff85ad25453765dd339b408c0087db1527 from 3.5.1 >>> (952fc18ef9ec707ebdc16c0786ec360295e5ff15 upstream) probably helps? >>> >>> From a quick look, I think that essentially has a : >>> >>> for (i = 0; i < ngroups; i++) { >>> >>> for (j = 0; j < ngroups; j++) { >>> >>> } >>> } >>> >>> type nested loop going on; for a filesystem this big it's going to take >>> almost >>> literally forever, if I read it right. >>> >>> -Eric >> >> Hello, >> >> It worked!! I can mount my filesystem now! >> >> I pulled down 3.5 and backed out that commit, I could not quickly find >> a doc to do this, so I will add how to do that below: >> >> 1. Clone Linux repo (3.5/stable as of this writing) >> git clone >> git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git >> >> 2. List commits: >> git log >> >> 3. Show a specific commit >> git show 8aeb00ff85ad25453765dd339b408c0087db1527 >> >> 4. How to revert the commit: >> git revert 8aeb00ff85ad25453765dd339b408c0087db1527 >> >> # On branch master >> nothing to commit (working directory clean) > > You didn't actually revert anything here, because your clone left > you on "master" branch, which points at 3.5 (i.e. 3.5.0). It does > not contain the commit which is of interest to you. > > - > linux-stable$git tag --contains 8aeb00ff > v3.5.1 > linux-stable$git branch --contains 8aeb00ff > linux-3.5.y > linux-stable$ > > > The master branch in linux-stable is left pointing at one of the > most recent mainline (i.e. non-stable) tags, and all of the stable > content is on individual branches (type "git branch" to see them). > > So if you do a "git checkout linux-3.5.y" and then do the revert, > you will actually be testing what you wanted to test. > > Paul. > -- Yikes, I saw the git details (via get show) but that must check the commit via git/inet-- I assumed that was also in the 3.5 tree, but its not per your check, so I've made some changes to my notes, recompiled, rebooted .. and success!! Woohoo! One other item, I've never used a git updated kernel before, usually just patch -p1 the mainline or pull it down directly (but git seems nicer now that I know how to do it), does the '+' signify its the 3.5.1 kernel and then a '+' because I made changes to it? p34:~# df -h | grep /r1 /dev/sda161T 16T 45T 26% /r1 p34:~# uname -a Linux p34 3.5.1+ #3 SMP Sun Aug 12 10:31:34 EDT 2012 x86_64 GNU/Linux p34:~# uptime 10:35:12 up 1 min, 1 user, load average: 0.05, 0.03, 0.01 p34:~# --- Updated notes: 1. Clone Linux repo (3.5/stable as of this writing) git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git linux-3.5 2.0 Cd into linux-3.5 cd linux-3.5 2.1 Check available kernel versions: git tag | tail -n 3 v3.5-rc6 v3.5-rc7 v3.5.1 2.2 Update to the latest 3.5.1 kernel: git checkout linux-3.5.1 Note: checking out 'v3.5.1'. .. HEAD is now at cbd3c20... Linux 3.5.1 2.3 Confirm it is 3.5.1: # head -n 3 Makefile VERSION = 3 PATCHLEVEL = 5 SUBLEVEL = 1 2.4 List commits: git log 3. Show a specific commit git show 8aeb00ff85ad25453765dd339b408c0087db1527 4. How to revert the commit: git revert 8aeb00ff85ad25453765dd339b408c0087db1527 (It brings up a text editor like svn/cvs, commit -> write/save/quit) [detached HEAD 35d699f] Revert "ext4: fix overhead calculation used by ext4_statfs()" Committer: root Your name and email address were configured automatically based on your username and hostname. Please check that they are accurate. You can suppress this message by setting them explicitly: git config --global user.name "Your Name" git config --global user.email y...@example.com After doing this, you may fix the identity used for this commit with: git commit --amend --reset-author 4 files changed, 57 insertions(+), 132 deletions(-) 5. Recompile, reboot, does it work, still? # df -h Filesystem Size Used Avail Use% Mounted on /dev/sda161T 17T 44T 28% /r1 # uname -a Linux p34 3.5.0 #1 SMP Sun Aug 12 09:42:41 EDT 2012 x86_64 GNU/Linux Yes! -- How to find where a particular commit lies? - linux-stable$git tag --contains 8aeb00ff v3.5.1 linux-stable$git branch --contains 8aeb00ff linux-3.5.y linux-stable$ -- Justin. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Upgraded from 3.4 to 3.5.1 kernel: machine does not boot
On Sun, Aug 12, 2012 at 9:51 AM, Justin Piszcz wrote: > On Sun, Aug 12, 2012 at 9:10 AM, Eric Sandeen wrote: >> On 8/10/12 11:14 PM, Justin Piszcz wrote: >>> On Fri, Aug 10, 2012 at 7:07 PM, Justin Piszcz Hi, Found the root cause, the 3.5.1 kernel cannot mount my ext4 filesystem (60TB). >> >> You are a brave man running ext4 at 60T, but thank you for testing :) >> >> Backing out 8aeb00ff85ad25453765dd339b408c0087db1527 from 3.5.1 >> (952fc18ef9ec707ebdc16c0786ec360295e5ff15 upstream) probably helps? >> >> From a quick look, I think that essentially has a : >> >> for (i = 0; i < ngroups; i++) { >> >> for (j = 0; j < ngroups; j++) { >> >> } >> } >> >> type nested loop going on; for a filesystem this big it's going to take >> almost >> literally forever, if I read it right. >> >> -Eric > > Hello, > > It worked!! I can mount my filesystem now! > > I pulled down 3.5 and backed out that commit, I could not quickly find > a doc to do this, so I will add how to do that below: > > 1. Clone Linux repo (3.5/stable as of this writing) > git clone > git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git > > 2. List commits: > git log > > 3. Show a specific commit > git show 8aeb00ff85ad25453765dd339b408c0087db1527 > > 4. How to revert the commit: > git revert 8aeb00ff85ad25453765dd339b408c0087db1527 > > # On branch master > nothing to commit (working directory clean) You didn't actually revert anything here, because your clone left you on "master" branch, which points at 3.5 (i.e. 3.5.0). It does not contain the commit which is of interest to you. - linux-stable$git tag --contains 8aeb00ff v3.5.1 linux-stable$git branch --contains 8aeb00ff linux-3.5.y linux-stable$ The master branch in linux-stable is left pointing at one of the most recent mainline (i.e. non-stable) tags, and all of the stable content is on individual branches (type "git branch" to see them). So if you do a "git checkout linux-3.5.y" and then do the revert, you will actually be testing what you wanted to test. Paul. -- > > 5. Recompile, reboot, does it work? > # df -h > Filesystem Size Used Avail Use% Mounted on > /dev/sda161T 17T 44T 28% /r1 > # uname -a > Linux p34 3.5.0 #1 SMP Sun Aug 12 09:42:41 EDT 2012 x86_64 GNU/Linux > > Yes! > > CC: Greg to see if this can be backed out for 3.5.2? > > Justin. > -- > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Upgraded from 3.4 to 3.5.1 kernel: machine does not boot
On Sun, Aug 12, 2012 at 9:10 AM, Eric Sandeen wrote: > On 8/10/12 11:14 PM, Justin Piszcz wrote: >> On Fri, Aug 10, 2012 at 7:07 PM, Justin Piszcz >>> >>> Hi, >>> >>> Found the root cause, the 3.5.1 kernel cannot mount my ext4 filesystem >>> (60TB). > > You are a brave man running ext4 at 60T, but thank you for testing :) > > Backing out 8aeb00ff85ad25453765dd339b408c0087db1527 from 3.5.1 > (952fc18ef9ec707ebdc16c0786ec360295e5ff15 upstream) probably helps? > > From a quick look, I think that essentially has a : > > for (i = 0; i < ngroups; i++) { > > for (j = 0; j < ngroups; j++) { > > } > } > > type nested loop going on; for a filesystem this big it's going to take almost > literally forever, if I read it right. > > -Eric Hello, It worked!! I can mount my filesystem now! I pulled down 3.5 and backed out that commit, I could not quickly find a doc to do this, so I will add how to do that below: 1. Clone Linux repo (3.5/stable as of this writing) git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git 2. List commits: git log 3. Show a specific commit git show 8aeb00ff85ad25453765dd339b408c0087db1527 4. How to revert the commit: git revert 8aeb00ff85ad25453765dd339b408c0087db1527 # On branch master nothing to commit (working directory clean) 5. Recompile, reboot, does it work? # df -h Filesystem Size Used Avail Use% Mounted on /dev/sda161T 17T 44T 28% /r1 # uname -a Linux p34 3.5.0 #1 SMP Sun Aug 12 09:42:41 EDT 2012 x86_64 GNU/Linux Yes! CC: Greg to see if this can be backed out for 3.5.2? Justin. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Upgraded from 3.4 to 3.5.1 kernel: machine does not boot
On 8/10/12 11:14 PM, Justin Piszcz wrote: > On Fri, Aug 10, 2012 at 7:07 PM, Justin Piszcz >> >> Hi, >> >> Found the root cause, the 3.5.1 kernel cannot mount my ext4 filesystem >> (60TB). You are a brave man running ext4 at 60T, but thank you for testing :) Backing out 8aeb00ff85ad25453765dd339b408c0087db1527 from 3.5.1 (952fc18ef9ec707ebdc16c0786ec360295e5ff15 upstream) probably helps? >From a quick look, I think that essentially has a : for (i = 0; i < ngroups; i++) { for (j = 0; j < ngroups; j++) { } } type nested loop going on; for a filesystem this big it's going to take almost literally forever, if I read it right. -Eric >> The 3.4 kernel works fine. >> >> This is proven by commenting out the filesystem in /etc/fstab with >> 3.5.1, and all is OK. >> >> -- >> >> Hi again, >> >> I tested with linux-3.6-rc1: >> >> The same problem, here is what I get from the strace: >> >> irectory) >> 4434 readlink("/dev", 0x7fff3b05c670, 4096) = -1 EINVAL (Invalid argument) >> 4434 readlink("/dev/sda1", 0x7fff3b05c670, 4096) = -1 EINVAL (Invalid >> argument) >> 4434 readlink("/r1", 0x7fff3b05c670, 4096) = -1 EINVAL (Invalid argument) >> 4434 getuid() = 0 >> 4434 geteuid() = 0 >> 4434 getgid() = 0 >> 4434 getegid() = 0 >> 4434 prctl(PR_GET_DUMPABLE)= 1 >> 4434 lstat("/etc/mtab", {st_mode=S_IFLNK|0777, st_size=12, ...}) = 0 >> 4434 getuid() = 0 >> 4434 geteuid() = 0 >> 4434 getgid() = 0 >> 4434 getegid() = 0 >> 4434 prctl(PR_GET_DUMPABLE)= 1 >> 4434 stat("/run", {st_mode=S_IFDIR|0755, st_size=820, ...}) = 0 >> 4434 lstat("/run/mount/utab", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0 >> 4434 open("/run/mount/utab", O_RDWR|O_CREAT, 0644) = 3 >> 4434 close(3) = 0 >> 4434 mount("/dev/sda1", "/r1", "ext4", MS_MGC_VAL|MS_NOATIME, NULL >> >> -- >> >> (w/ 3.6-rc1) >> >> [ 89.868843] mount R running task0 4434 4433 >> 0x0009 >> [ 89.868847] 880c246b7b68 816c9279 880c246b7aa8 >> 880c246b7fd8 >> [ 89.868851] 880c246b7fd8 4000 88062720cdb0 >> 880c246862d0 >> [ 89.868855] 000116c0 880623a863c0 880623a863c0 >> >> [ 89.868855] Call Trace: >> [ 89.868858] [] ? __schedule+0x299/0x770 >> [ 89.868860] [] ? __schedule+0x299/0x770 >> [ 89.868864] [] ? ext4_get_group_desc+0x49/0xb0 >> [ 89.868868] [] ? ext4_calculate_overhead+0x131/0x3e0 >> [ 89.868871] [] ? ext4_fill_super+0x1a4b/0x28d0 >> [ 89.868875] [] ? mount_bdev+0x1a1/0x1e0 >> [ 89.868877] [] ? ext4_calculate_overhead+0x3e0/0x3e0 >> [ 89.868880] [] ? ext4_mount+0x10/0x20 >> [ 89.868882] [] ? mount_fs+0x1b/0xd0 >> [ 89.868885] [] ? vfs_kern_mount+0x6f/0x110 >> [ 89.86] [] ? do_kern_mount+0x4f/0x100 >> [ 89.868890] [] ? do_mount+0x2fe/0x8a0 >> [ 89.868894] [] ? strndup_user+0x53/0x70 >> [ 89.868896] [] ? sys_mount+0x90/0xe0 >> [ 89.868899] [] ? tracesys+0xd4/0xd9 >> >> Justin. >> >> >> > > CC: linux-ext4 > > Any ideas here (kernel 3.4 and below can mount 60TB ext4 no issues) > but > 3.5.1 (did not try 3.5) cannot mount the filesystem. > > Justin. > > Justin. > -- > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Upgraded from 3.4 to 3.5.1 kernel: machine does not boot
On 8/10/12 11:14 PM, Justin Piszcz wrote: On Fri, Aug 10, 2012 at 7:07 PM, Justin Piszcz Hi, Found the root cause, the 3.5.1 kernel cannot mount my ext4 filesystem (60TB). You are a brave man running ext4 at 60T, but thank you for testing :) Backing out 8aeb00ff85ad25453765dd339b408c0087db1527 from 3.5.1 (952fc18ef9ec707ebdc16c0786ec360295e5ff15 upstream) probably helps? From a quick look, I think that essentially has a : for (i = 0; i ngroups; i++) { for (j = 0; j ngroups; j++) { } } type nested loop going on; for a filesystem this big it's going to take almost literally forever, if I read it right. -Eric The 3.4 kernel works fine. This is proven by commenting out the filesystem in /etc/fstab with 3.5.1, and all is OK. -- Hi again, I tested with linux-3.6-rc1: The same problem, here is what I get from the strace: irectory) 4434 readlink(/dev, 0x7fff3b05c670, 4096) = -1 EINVAL (Invalid argument) 4434 readlink(/dev/sda1, 0x7fff3b05c670, 4096) = -1 EINVAL (Invalid argument) 4434 readlink(/r1, 0x7fff3b05c670, 4096) = -1 EINVAL (Invalid argument) 4434 getuid() = 0 4434 geteuid() = 0 4434 getgid() = 0 4434 getegid() = 0 4434 prctl(PR_GET_DUMPABLE)= 1 4434 lstat(/etc/mtab, {st_mode=S_IFLNK|0777, st_size=12, ...}) = 0 4434 getuid() = 0 4434 geteuid() = 0 4434 getgid() = 0 4434 getegid() = 0 4434 prctl(PR_GET_DUMPABLE)= 1 4434 stat(/run, {st_mode=S_IFDIR|0755, st_size=820, ...}) = 0 4434 lstat(/run/mount/utab, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0 4434 open(/run/mount/utab, O_RDWR|O_CREAT, 0644) = 3 4434 close(3) = 0 4434 mount(/dev/sda1, /r1, ext4, MS_MGC_VAL|MS_NOATIME, NULL -- (w/ 3.6-rc1) [ 89.868843] mount R running task0 4434 4433 0x0009 [ 89.868847] 880c246b7b68 816c9279 880c246b7aa8 880c246b7fd8 [ 89.868851] 880c246b7fd8 4000 88062720cdb0 880c246862d0 [ 89.868855] 000116c0 880623a863c0 880623a863c0 [ 89.868855] Call Trace: [ 89.868858] [816c9279] ? __schedule+0x299/0x770 [ 89.868860] [816c9279] ? __schedule+0x299/0x770 [ 89.868864] [8114a729] ? ext4_get_group_desc+0x49/0xb0 [ 89.868868] [81161d41] ? ext4_calculate_overhead+0x131/0x3e0 [ 89.868871] [81163a3b] ? ext4_fill_super+0x1a4b/0x28d0 [ 89.868875] [810cc301] ? mount_bdev+0x1a1/0x1e0 [ 89.868877] [81161ff0] ? ext4_calculate_overhead+0x3e0/0x3e0 [ 89.868880] [8115dd00] ? ext4_mount+0x10/0x20 [ 89.868882] [810cc55b] ? mount_fs+0x1b/0xd0 [ 89.868885] [810e57af] ? vfs_kern_mount+0x6f/0x110 [ 89.86] [810e58cf] ? do_kern_mount+0x4f/0x100 [ 89.868890] [810e6dae] ? do_mount+0x2fe/0x8a0 [ 89.868894] [8109c0a3] ? strndup_user+0x53/0x70 [ 89.868896] [810e73e0] ? sys_mount+0x90/0xe0 [ 89.868899] [816cafa1] ? tracesys+0xd4/0xd9 Justin. CC: linux-ext4 Any ideas here (kernel 3.4 and below can mount 60TB ext4 no issues) but 3.5.1 (did not try 3.5) cannot mount the filesystem. Justin. Justin. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Upgraded from 3.4 to 3.5.1 kernel: machine does not boot
On Sun, Aug 12, 2012 at 9:10 AM, Eric Sandeen sand...@sandeen.net wrote: On 8/10/12 11:14 PM, Justin Piszcz wrote: On Fri, Aug 10, 2012 at 7:07 PM, Justin Piszcz Hi, Found the root cause, the 3.5.1 kernel cannot mount my ext4 filesystem (60TB). You are a brave man running ext4 at 60T, but thank you for testing :) Backing out 8aeb00ff85ad25453765dd339b408c0087db1527 from 3.5.1 (952fc18ef9ec707ebdc16c0786ec360295e5ff15 upstream) probably helps? From a quick look, I think that essentially has a : for (i = 0; i ngroups; i++) { for (j = 0; j ngroups; j++) { } } type nested loop going on; for a filesystem this big it's going to take almost literally forever, if I read it right. -Eric Hello, It worked!! I can mount my filesystem now! I pulled down 3.5 and backed out that commit, I could not quickly find a doc to do this, so I will add how to do that below: 1. Clone Linux repo (3.5/stable as of this writing) git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git 2. List commits: git log 3. Show a specific commit git show 8aeb00ff85ad25453765dd339b408c0087db1527 4. How to revert the commit: git revert 8aeb00ff85ad25453765dd339b408c0087db1527 # On branch master nothing to commit (working directory clean) 5. Recompile, reboot, does it work? # df -h Filesystem Size Used Avail Use% Mounted on /dev/sda161T 17T 44T 28% /r1 # uname -a Linux p34 3.5.0 #1 SMP Sun Aug 12 09:42:41 EDT 2012 x86_64 GNU/Linux Yes! CC: Greg to see if this can be backed out for 3.5.2? Justin. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Upgraded from 3.4 to 3.5.1 kernel: machine does not boot
On Sun, Aug 12, 2012 at 9:51 AM, Justin Piszcz jpis...@lucidpixels.com wrote: On Sun, Aug 12, 2012 at 9:10 AM, Eric Sandeen sand...@sandeen.net wrote: On 8/10/12 11:14 PM, Justin Piszcz wrote: On Fri, Aug 10, 2012 at 7:07 PM, Justin Piszcz Hi, Found the root cause, the 3.5.1 kernel cannot mount my ext4 filesystem (60TB). You are a brave man running ext4 at 60T, but thank you for testing :) Backing out 8aeb00ff85ad25453765dd339b408c0087db1527 from 3.5.1 (952fc18ef9ec707ebdc16c0786ec360295e5ff15 upstream) probably helps? From a quick look, I think that essentially has a : for (i = 0; i ngroups; i++) { for (j = 0; j ngroups; j++) { } } type nested loop going on; for a filesystem this big it's going to take almost literally forever, if I read it right. -Eric Hello, It worked!! I can mount my filesystem now! I pulled down 3.5 and backed out that commit, I could not quickly find a doc to do this, so I will add how to do that below: 1. Clone Linux repo (3.5/stable as of this writing) git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git 2. List commits: git log 3. Show a specific commit git show 8aeb00ff85ad25453765dd339b408c0087db1527 4. How to revert the commit: git revert 8aeb00ff85ad25453765dd339b408c0087db1527 # On branch master nothing to commit (working directory clean) You didn't actually revert anything here, because your clone left you on master branch, which points at 3.5 (i.e. 3.5.0). It does not contain the commit which is of interest to you. - linux-stable$git tag --contains 8aeb00ff v3.5.1 linux-stable$git branch --contains 8aeb00ff linux-3.5.y linux-stable$ The master branch in linux-stable is left pointing at one of the most recent mainline (i.e. non-stable) tags, and all of the stable content is on individual branches (type git branch to see them). So if you do a git checkout linux-3.5.y and then do the revert, you will actually be testing what you wanted to test. Paul. -- 5. Recompile, reboot, does it work? # df -h Filesystem Size Used Avail Use% Mounted on /dev/sda161T 17T 44T 28% /r1 # uname -a Linux p34 3.5.0 #1 SMP Sun Aug 12 09:42:41 EDT 2012 x86_64 GNU/Linux Yes! CC: Greg to see if this can be backed out for 3.5.2? Justin. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Upgraded from 3.4 to 3.5.1 kernel: machine does not boot
On Sun, Aug 12, 2012 at 10:13 AM, Paul Gortmaker paul.gortma...@windriver.com wrote: On Sun, Aug 12, 2012 at 9:51 AM, Justin Piszcz jpis...@lucidpixels.com wrote: On Sun, Aug 12, 2012 at 9:10 AM, Eric Sandeen sand...@sandeen.net wrote: On 8/10/12 11:14 PM, Justin Piszcz wrote: On Fri, Aug 10, 2012 at 7:07 PM, Justin Piszcz Hi, Found the root cause, the 3.5.1 kernel cannot mount my ext4 filesystem (60TB). You are a brave man running ext4 at 60T, but thank you for testing :) Backing out 8aeb00ff85ad25453765dd339b408c0087db1527 from 3.5.1 (952fc18ef9ec707ebdc16c0786ec360295e5ff15 upstream) probably helps? From a quick look, I think that essentially has a : for (i = 0; i ngroups; i++) { for (j = 0; j ngroups; j++) { } } type nested loop going on; for a filesystem this big it's going to take almost literally forever, if I read it right. -Eric Hello, It worked!! I can mount my filesystem now! I pulled down 3.5 and backed out that commit, I could not quickly find a doc to do this, so I will add how to do that below: 1. Clone Linux repo (3.5/stable as of this writing) git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git 2. List commits: git log 3. Show a specific commit git show 8aeb00ff85ad25453765dd339b408c0087db1527 4. How to revert the commit: git revert 8aeb00ff85ad25453765dd339b408c0087db1527 # On branch master nothing to commit (working directory clean) You didn't actually revert anything here, because your clone left you on master branch, which points at 3.5 (i.e. 3.5.0). It does not contain the commit which is of interest to you. - linux-stable$git tag --contains 8aeb00ff v3.5.1 linux-stable$git branch --contains 8aeb00ff linux-3.5.y linux-stable$ The master branch in linux-stable is left pointing at one of the most recent mainline (i.e. non-stable) tags, and all of the stable content is on individual branches (type git branch to see them). So if you do a git checkout linux-3.5.y and then do the revert, you will actually be testing what you wanted to test. Paul. -- Yikes, I saw the git details (via get show) but that must check the commit via git/inet-- I assumed that was also in the 3.5 tree, but its not per your check, so I've made some changes to my notes, recompiled, rebooted .. and success!! Woohoo! One other item, I've never used a git updated kernel before, usually just patch -p1 the mainline or pull it down directly (but git seems nicer now that I know how to do it), does the '+' signify its the 3.5.1 kernel and then a '+' because I made changes to it? p34:~# df -h | grep /r1 /dev/sda161T 16T 45T 26% /r1 p34:~# uname -a Linux p34 3.5.1+ #3 SMP Sun Aug 12 10:31:34 EDT 2012 x86_64 GNU/Linux p34:~# uptime 10:35:12 up 1 min, 1 user, load average: 0.05, 0.03, 0.01 p34:~# --- Updated notes: 1. Clone Linux repo (3.5/stable as of this writing) git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git linux-3.5 2.0 Cd into linux-3.5 cd linux-3.5 2.1 Check available kernel versions: git tag | tail -n 3 v3.5-rc6 v3.5-rc7 v3.5.1 2.2 Update to the latest 3.5.1 kernel: git checkout linux-3.5.1 Note: checking out 'v3.5.1'. .. HEAD is now at cbd3c20... Linux 3.5.1 2.3 Confirm it is 3.5.1: # head -n 3 Makefile VERSION = 3 PATCHLEVEL = 5 SUBLEVEL = 1 2.4 List commits: git log 3. Show a specific commit git show 8aeb00ff85ad25453765dd339b408c0087db1527 4. How to revert the commit: git revert 8aeb00ff85ad25453765dd339b408c0087db1527 (It brings up a text editor like svn/cvs, commit - write/save/quit) [detached HEAD 35d699f] Revert ext4: fix overhead calculation used by ext4_statfs() Committer: root r...@lucidpixels.com Your name and email address were configured automatically based on your username and hostname. Please check that they are accurate. You can suppress this message by setting them explicitly: git config --global user.name Your Name git config --global user.email y...@example.com After doing this, you may fix the identity used for this commit with: git commit --amend --reset-author 4 files changed, 57 insertions(+), 132 deletions(-) 5. Recompile, reboot, does it work, still? # df -h Filesystem Size Used Avail Use% Mounted on /dev/sda161T 17T 44T 28% /r1 # uname -a Linux p34 3.5.0 #1 SMP Sun Aug 12 09:42:41 EDT 2012 x86_64 GNU/Linux Yes! -- How to find where a particular commit lies? - linux-stable$git tag --contains 8aeb00ff v3.5.1 linux-stable$git branch --contains 8aeb00ff linux-3.5.y linux-stable$ -- Justin. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Upgraded from 3.4 to 3.5.1 kernel: machine does not boot
On Fri, Aug 10, 2012 at 7:07 PM, Justin Piszcz > > Hi, > > Found the root cause, the 3.5.1 kernel cannot mount my ext4 filesystem > (60TB). > > The 3.4 kernel works fine. > > This is proven by commenting out the filesystem in /etc/fstab with > 3.5.1, and all is OK. > > -- > > Hi again, > > I tested with linux-3.6-rc1: > > The same problem, here is what I get from the strace: > > irectory) > 4434 readlink("/dev", 0x7fff3b05c670, 4096) = -1 EINVAL (Invalid argument) > 4434 readlink("/dev/sda1", 0x7fff3b05c670, 4096) = -1 EINVAL (Invalid > argument) > 4434 readlink("/r1", 0x7fff3b05c670, 4096) = -1 EINVAL (Invalid argument) > 4434 getuid() = 0 > 4434 geteuid() = 0 > 4434 getgid() = 0 > 4434 getegid() = 0 > 4434 prctl(PR_GET_DUMPABLE)= 1 > 4434 lstat("/etc/mtab", {st_mode=S_IFLNK|0777, st_size=12, ...}) = 0 > 4434 getuid() = 0 > 4434 geteuid() = 0 > 4434 getgid() = 0 > 4434 getegid() = 0 > 4434 prctl(PR_GET_DUMPABLE)= 1 > 4434 stat("/run", {st_mode=S_IFDIR|0755, st_size=820, ...}) = 0 > 4434 lstat("/run/mount/utab", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0 > 4434 open("/run/mount/utab", O_RDWR|O_CREAT, 0644) = 3 > 4434 close(3) = 0 > 4434 mount("/dev/sda1", "/r1", "ext4", MS_MGC_VAL|MS_NOATIME, NULL > > -- > > (w/ 3.6-rc1) > > [ 89.868843] mount R running task0 4434 4433 > 0x0009 > [ 89.868847] 880c246b7b68 816c9279 880c246b7aa8 > 880c246b7fd8 > [ 89.868851] 880c246b7fd8 4000 88062720cdb0 > 880c246862d0 > [ 89.868855] 000116c0 880623a863c0 880623a863c0 > > [ 89.868855] Call Trace: > [ 89.868858] [] ? __schedule+0x299/0x770 > [ 89.868860] [] ? __schedule+0x299/0x770 > [ 89.868864] [] ? ext4_get_group_desc+0x49/0xb0 > [ 89.868868] [] ? ext4_calculate_overhead+0x131/0x3e0 > [ 89.868871] [] ? ext4_fill_super+0x1a4b/0x28d0 > [ 89.868875] [] ? mount_bdev+0x1a1/0x1e0 > [ 89.868877] [] ? ext4_calculate_overhead+0x3e0/0x3e0 > [ 89.868880] [] ? ext4_mount+0x10/0x20 > [ 89.868882] [] ? mount_fs+0x1b/0xd0 > [ 89.868885] [] ? vfs_kern_mount+0x6f/0x110 > [ 89.86] [] ? do_kern_mount+0x4f/0x100 > [ 89.868890] [] ? do_mount+0x2fe/0x8a0 > [ 89.868894] [] ? strndup_user+0x53/0x70 > [ 89.868896] [] ? sys_mount+0x90/0xe0 > [ 89.868899] [] ? tracesys+0xd4/0xd9 > > Justin. > > > CC: linux-ext4 Any ideas here (kernel 3.4 and below can mount 60TB ext4 no issues) but > 3.5.1 (did not try 3.5) cannot mount the filesystem. Justin. Justin. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: Upgraded from 3.4 to 3.5.1 kernel: machine does not boot
-Original Message- From: Justin Piszcz [mailto:jpis...@lucidpixels.com] Sent: Friday, August 10, 2012 5:46 PM To: Jesper Juhl Cc: linux-kernel@vger.kernel.org; a...@solarrain.com Subject: Re: Upgraded from 3.4 to 3.5.1 kernel: machine does not boot On Fri, Aug 10, 2012 at 1:53 PM, Jesper Juhl wrote: > On Fri, 10 Aug 2012, Justin Piszcz wrote: > >> Hello, >> >> Motherboard: Supermicro X8DTH-6F >> Distro: Debian Testing x86_64 >> >> >From 3.4 -> 3.5.1 on x86_64 make oldconfig and a few minor changes and the >> machine attempts to boot but hangs at the filesystem mounting part of the >> boot process. Hi, Found the root cause, the 3.5.1 kernel cannot mount my ext4 filesystem (60TB). The 3.4 kernel works fine. This is proven by commenting out the filesystem in /etc/fstab with 3.5.1, and all is OK. -- Hi again, I tested with linux-3.6-rc1: The same problem, here is what I get from the strace: irectory) 4434 readlink("/dev", 0x7fff3b05c670, 4096) = -1 EINVAL (Invalid argument) 4434 readlink("/dev/sda1", 0x7fff3b05c670, 4096) = -1 EINVAL (Invalid argument) 4434 readlink("/r1", 0x7fff3b05c670, 4096) = -1 EINVAL (Invalid argument) 4434 getuid() = 0 4434 geteuid() = 0 4434 getgid() = 0 4434 getegid() = 0 4434 prctl(PR_GET_DUMPABLE)= 1 4434 lstat("/etc/mtab", {st_mode=S_IFLNK|0777, st_size=12, ...}) = 0 4434 getuid() = 0 4434 geteuid() = 0 4434 getgid() = 0 4434 getegid() = 0 4434 prctl(PR_GET_DUMPABLE)= 1 4434 stat("/run", {st_mode=S_IFDIR|0755, st_size=820, ...}) = 0 4434 lstat("/run/mount/utab", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0 4434 open("/run/mount/utab", O_RDWR|O_CREAT, 0644) = 3 4434 close(3) = 0 4434 mount("/dev/sda1", "/r1", "ext4", MS_MGC_VAL|MS_NOATIME, NULL -- (w/ 3.6-rc1) [ 89.868843] mount R running task0 4434 4433 0x0009 [ 89.868847] 880c246b7b68 816c9279 880c246b7aa8 880c246b7fd8 [ 89.868851] 880c246b7fd8 4000 88062720cdb0 880c246862d0 [ 89.868855] 000116c0 880623a863c0 880623a863c0 [ 89.868855] Call Trace: [ 89.868858] [] ? __schedule+0x299/0x770 [ 89.868860] [] ? __schedule+0x299/0x770 [ 89.868864] [] ? ext4_get_group_desc+0x49/0xb0 [ 89.868868] [] ? ext4_calculate_overhead+0x131/0x3e0 [ 89.868871] [] ? ext4_fill_super+0x1a4b/0x28d0 [ 89.868875] [] ? mount_bdev+0x1a1/0x1e0 [ 89.868877] [] ? ext4_calculate_overhead+0x3e0/0x3e0 [ 89.868880] [] ? ext4_mount+0x10/0x20 [ 89.868882] [] ? mount_fs+0x1b/0xd0 [ 89.868885] [] ? vfs_kern_mount+0x6f/0x110 [ 89.86] [] ? do_kern_mount+0x4f/0x100 [ 89.868890] [] ? do_mount+0x2fe/0x8a0 [ 89.868894] [] ? strndup_user+0x53/0x70 [ 89.868896] [] ? sys_mount+0x90/0xe0 [ 89.868899] [] ? tracesys+0xd4/0xd9 Justin. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Upgraded from 3.4 to 3.5.1 kernel: machine does not boot
On Fri, Aug 10, 2012 at 1:53 PM, Jesper Juhl wrote: > On Fri, 10 Aug 2012, Justin Piszcz wrote: > >> Hello, >> >> Motherboard: Supermicro X8DTH-6F >> Distro: Debian Testing x86_64 >> >> >From 3.4 -> 3.5.1 on x86_64 make oldconfig and a few minor changes and the >> machine attempts to boot but hangs at the filesystem mounting part of the >> boot process. Hi, Found the root cause, the 3.5.1 kernel cannot mount my ext4 filesystem (60TB). The 3.4 kernel works fine. This is proven by commenting out the filesystem in /etc/fstab with 3.5.1, and all is OK. When I run mount for that filesystem, it hangs, I ran alt+sysrq+t to get additional output and I have pasted it below with the 3.5.1 kernel: [ 160.373406] mount R running task0 4361 4355 0x [ 160.373407] 8806266bdb68 0086 8806266bdaa8 8806266bdfd8 [ 160.373410] 8806266bdfd8 4000 8806270b0600 880626c73a10 [ 160.373413] 00011240 880c260177c0 880c260177c0 [ 160.373415] Call Trace: [ 160.373416] [] ? __schedule+0x299/0x770 [ 160.373418] [] __cond_resched+0x25/0x40 [ 160.373420] [] _cond_resched+0x2a/0x40 [ 160.373421] [] ext4_calculate_overhead+0x239/0x3e0 [ 160.373425] [] ext4_fill_super+0x1aa9/0x2930 [ 160.373427] [] mount_bdev+0x19f/0x1e0 [ 160.373429] [] ? ext4_calculate_overhead+0x3e0/0x3e0 [ 160.373431] [] ext4_mount+0x10/0x20 [ 160.373433] [] mount_fs+0x1b/0xd0 [ 160.373434] [] vfs_kern_mount+0x6f/0x110 [ 160.373437] [] do_kern_mount+0x4f/0x100 [ 160.373439] [] do_mount+0x2fe/0x8a0 [ 160.373440] [] ? strndup_user+0x53/0x70 [ 160.373442] [] sys_mount+0x90/0xe0 [ 160.373443] [] system_call_fastpath+0x1a/0x1f [ 160.373446] jbd2/sda1-8 S 880c2675f800 0 4362 2 0x [ 160.373448] 880623ca9e50 0046 880626c73a10 880623ca9fd8 [ 160.373450] 880623ca9fd8 4000 8806271b9850 880626d08250 [ 160.373453] 880623ca9da0 8806266bdbe0 880c2675f8a0 880c2675f888 [ 160.373455] Call Trace: [ 160.373456] [] ? default_wake_function+0xd/0x10 [ 160.373458] [] ? autoremove_wake_function+0x11/0x40 [ 160.373460] [] ? __wake_up_common+0x55/0x90 [ 160.373462] [] schedule+0x24/0x70 [ 160.373463] [] kjournald2+0x1ce/0x1e0 [ 160.373465] [] ? abort_exclusive_wait+0xb0/0xb0 [ 160.373467] [] ? commit_timeout+0x10/0x10 [ 160.373469] [] kthread+0x8e/0xa0 [ 160.373471] [] kernel_thread_helper+0x4/0x10 [ 160.373472] [] ? kthread_flush_work_fn+0x10/0x10 [ 160.373474] [] ? gs_change+0xb/0xb Justin. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Upgraded from 3.4 to 3.5.1 kernel: machine does not boot
On Fri, 10 Aug 2012, Justin Piszcz wrote: > Hello, > > Motherboard: Supermicro X8DTH-6F > Distro: Debian Testing x86_64 > > >From 3.4 -> 3.5.1 on x86_64 make oldconfig and a few minor changes and the > machine attempts to boot but hangs at the filesystem mounting part of the > boot process. > > Picture of where it stops working (a little burry but readable) > http://home.comcast.net/~jpiszcz/20120810/3.5-kernel-hangs.jpg > > Kernel config 3.4 (working) > http://home.comcast.net/~jpiszcz/20120810/config-3.4.txt > > Kernel config 3.5.1 (hangs) > http://home.comcast.net/~jpiszcz/20120810/config-3.5.1.txt > > As you see towards the end the machine has been sitting there for 1 hour as > that's the timeout I have the drives spindown on the 3ware card. > > Any thoughts as what is wrong here? > Not really, but some (rather obvious) ideas on what to try: - Does 3.5 work? (could be that whatever broke things for you was introduced in 3.5.1). - Does the latest 3.4.8 work? - Does 3.6-rc1 (or even the latest snapshot of Linus' tree, post-rc1) work? - If noone comes up with a good idea as to the cause of your troubles, you could try bisecting between your last working kernel and 3.5.1 to try and narrow it down to one (or a few) commits that are causing your trouble. -- Jesper Juhlhttp://www.chaosbits.net/ Don't top-post http://www.catb.org/jargon/html/T/top-post.html Plain text mails only, please. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Upgraded from 3.4 to 3.5.1 kernel: machine does not boot
On Fri, 10 Aug 2012, Justin Piszcz wrote: Hello, Motherboard: Supermicro X8DTH-6F Distro: Debian Testing x86_64 From 3.4 - 3.5.1 on x86_64 make oldconfig and a few minor changes and the machine attempts to boot but hangs at the filesystem mounting part of the boot process. Picture of where it stops working (a little burry but readable) http://home.comcast.net/~jpiszcz/20120810/3.5-kernel-hangs.jpg Kernel config 3.4 (working) http://home.comcast.net/~jpiszcz/20120810/config-3.4.txt Kernel config 3.5.1 (hangs) http://home.comcast.net/~jpiszcz/20120810/config-3.5.1.txt As you see towards the end the machine has been sitting there for 1 hour as that's the timeout I have the drives spindown on the 3ware card. Any thoughts as what is wrong here? Not really, but some (rather obvious) ideas on what to try: - Does 3.5 work? (could be that whatever broke things for you was introduced in 3.5.1). - Does the latest 3.4.8 work? - Does 3.6-rc1 (or even the latest snapshot of Linus' tree, post-rc1) work? - If noone comes up with a good idea as to the cause of your troubles, you could try bisecting between your last working kernel and 3.5.1 to try and narrow it down to one (or a few) commits that are causing your trouble. -- Jesper Juhl j...@chaosbits.net http://www.chaosbits.net/ Don't top-post http://www.catb.org/jargon/html/T/top-post.html Plain text mails only, please. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Upgraded from 3.4 to 3.5.1 kernel: machine does not boot
On Fri, Aug 10, 2012 at 1:53 PM, Jesper Juhl j...@chaosbits.net wrote: On Fri, 10 Aug 2012, Justin Piszcz wrote: Hello, Motherboard: Supermicro X8DTH-6F Distro: Debian Testing x86_64 From 3.4 - 3.5.1 on x86_64 make oldconfig and a few minor changes and the machine attempts to boot but hangs at the filesystem mounting part of the boot process. Hi, Found the root cause, the 3.5.1 kernel cannot mount my ext4 filesystem (60TB). The 3.4 kernel works fine. This is proven by commenting out the filesystem in /etc/fstab with 3.5.1, and all is OK. When I run mount for that filesystem, it hangs, I ran alt+sysrq+t to get additional output and I have pasted it below with the 3.5.1 kernel: [ 160.373406] mount R running task0 4361 4355 0x [ 160.373407] 8806266bdb68 0086 8806266bdaa8 8806266bdfd8 [ 160.373410] 8806266bdfd8 4000 8806270b0600 880626c73a10 [ 160.373413] 00011240 880c260177c0 880c260177c0 [ 160.373415] Call Trace: [ 160.373416] [816bd009] ? __schedule+0x299/0x770 [ 160.373418] [81053465] __cond_resched+0x25/0x40 [ 160.373420] [816bd6ba] _cond_resched+0x2a/0x40 [ 160.373421] [8115bc09] ext4_calculate_overhead+0x239/0x3e0 [ 160.373425] [8115d859] ext4_fill_super+0x1aa9/0x2930 [ 160.373427] [810c677f] mount_bdev+0x19f/0x1e0 [ 160.373429] [8115bdb0] ? ext4_calculate_overhead+0x3e0/0x3e0 [ 160.373431] [811579c0] ext4_mount+0x10/0x20 [ 160.373433] [810c69eb] mount_fs+0x1b/0xd0 [ 160.373434] [810df2df] vfs_kern_mount+0x6f/0x110 [ 160.373437] [810df3ff] do_kern_mount+0x4f/0x100 [ 160.373439] [810e091e] do_mount+0x2fe/0x8a0 [ 160.373440] [81097053] ? strndup_user+0x53/0x70 [ 160.373442] [810e0fd0] sys_mount+0x90/0xe0 [ 160.373443] [816beba6] system_call_fastpath+0x1a/0x1f [ 160.373446] jbd2/sda1-8 S 880c2675f800 0 4362 2 0x [ 160.373448] 880623ca9e50 0046 880626c73a10 880623ca9fd8 [ 160.373450] 880623ca9fd8 4000 8806271b9850 880626d08250 [ 160.373453] 880623ca9da0 8806266bdbe0 880c2675f8a0 880c2675f888 [ 160.373455] Call Trace: [ 160.373456] [81055f9d] ? default_wake_function+0xd/0x10 [ 160.373458] [8104acf1] ? autoremove_wake_function+0x11/0x40 [ 160.373460] [810521f5] ? __wake_up_common+0x55/0x90 [ 160.373462] [816bd504] schedule+0x24/0x70 [ 160.373463] [81183f8e] kjournald2+0x1ce/0x1e0 [ 160.373465] [8104ace0] ? abort_exclusive_wait+0xb0/0xb0 [ 160.373467] [81183dc0] ? commit_timeout+0x10/0x10 [ 160.373469] [8104a27e] kthread+0x8e/0xa0 [ 160.373471] [816bfed4] kernel_thread_helper+0x4/0x10 [ 160.373472] [8104a1f0] ? kthread_flush_work_fn+0x10/0x10 [ 160.373474] [816bfed0] ? gs_change+0xb/0xb Justin. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: Upgraded from 3.4 to 3.5.1 kernel: machine does not boot
-Original Message- From: Justin Piszcz [mailto:jpis...@lucidpixels.com] Sent: Friday, August 10, 2012 5:46 PM To: Jesper Juhl Cc: linux-kernel@vger.kernel.org; a...@solarrain.com Subject: Re: Upgraded from 3.4 to 3.5.1 kernel: machine does not boot On Fri, Aug 10, 2012 at 1:53 PM, Jesper Juhl j...@chaosbits.net wrote: On Fri, 10 Aug 2012, Justin Piszcz wrote: Hello, Motherboard: Supermicro X8DTH-6F Distro: Debian Testing x86_64 From 3.4 - 3.5.1 on x86_64 make oldconfig and a few minor changes and the machine attempts to boot but hangs at the filesystem mounting part of the boot process. Hi, Found the root cause, the 3.5.1 kernel cannot mount my ext4 filesystem (60TB). The 3.4 kernel works fine. This is proven by commenting out the filesystem in /etc/fstab with 3.5.1, and all is OK. -- Hi again, I tested with linux-3.6-rc1: The same problem, here is what I get from the strace: irectory) 4434 readlink(/dev, 0x7fff3b05c670, 4096) = -1 EINVAL (Invalid argument) 4434 readlink(/dev/sda1, 0x7fff3b05c670, 4096) = -1 EINVAL (Invalid argument) 4434 readlink(/r1, 0x7fff3b05c670, 4096) = -1 EINVAL (Invalid argument) 4434 getuid() = 0 4434 geteuid() = 0 4434 getgid() = 0 4434 getegid() = 0 4434 prctl(PR_GET_DUMPABLE)= 1 4434 lstat(/etc/mtab, {st_mode=S_IFLNK|0777, st_size=12, ...}) = 0 4434 getuid() = 0 4434 geteuid() = 0 4434 getgid() = 0 4434 getegid() = 0 4434 prctl(PR_GET_DUMPABLE)= 1 4434 stat(/run, {st_mode=S_IFDIR|0755, st_size=820, ...}) = 0 4434 lstat(/run/mount/utab, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0 4434 open(/run/mount/utab, O_RDWR|O_CREAT, 0644) = 3 4434 close(3) = 0 4434 mount(/dev/sda1, /r1, ext4, MS_MGC_VAL|MS_NOATIME, NULL -- (w/ 3.6-rc1) [ 89.868843] mount R running task0 4434 4433 0x0009 [ 89.868847] 880c246b7b68 816c9279 880c246b7aa8 880c246b7fd8 [ 89.868851] 880c246b7fd8 4000 88062720cdb0 880c246862d0 [ 89.868855] 000116c0 880623a863c0 880623a863c0 [ 89.868855] Call Trace: [ 89.868858] [816c9279] ? __schedule+0x299/0x770 [ 89.868860] [816c9279] ? __schedule+0x299/0x770 [ 89.868864] [8114a729] ? ext4_get_group_desc+0x49/0xb0 [ 89.868868] [81161d41] ? ext4_calculate_overhead+0x131/0x3e0 [ 89.868871] [81163a3b] ? ext4_fill_super+0x1a4b/0x28d0 [ 89.868875] [810cc301] ? mount_bdev+0x1a1/0x1e0 [ 89.868877] [81161ff0] ? ext4_calculate_overhead+0x3e0/0x3e0 [ 89.868880] [8115dd00] ? ext4_mount+0x10/0x20 [ 89.868882] [810cc55b] ? mount_fs+0x1b/0xd0 [ 89.868885] [810e57af] ? vfs_kern_mount+0x6f/0x110 [ 89.86] [810e58cf] ? do_kern_mount+0x4f/0x100 [ 89.868890] [810e6dae] ? do_mount+0x2fe/0x8a0 [ 89.868894] [8109c0a3] ? strndup_user+0x53/0x70 [ 89.868896] [810e73e0] ? sys_mount+0x90/0xe0 [ 89.868899] [816cafa1] ? tracesys+0xd4/0xd9 Justin. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Upgraded from 3.4 to 3.5.1 kernel: machine does not boot
On Fri, Aug 10, 2012 at 7:07 PM, Justin Piszcz Hi, Found the root cause, the 3.5.1 kernel cannot mount my ext4 filesystem (60TB). The 3.4 kernel works fine. This is proven by commenting out the filesystem in /etc/fstab with 3.5.1, and all is OK. -- Hi again, I tested with linux-3.6-rc1: The same problem, here is what I get from the strace: irectory) 4434 readlink(/dev, 0x7fff3b05c670, 4096) = -1 EINVAL (Invalid argument) 4434 readlink(/dev/sda1, 0x7fff3b05c670, 4096) = -1 EINVAL (Invalid argument) 4434 readlink(/r1, 0x7fff3b05c670, 4096) = -1 EINVAL (Invalid argument) 4434 getuid() = 0 4434 geteuid() = 0 4434 getgid() = 0 4434 getegid() = 0 4434 prctl(PR_GET_DUMPABLE)= 1 4434 lstat(/etc/mtab, {st_mode=S_IFLNK|0777, st_size=12, ...}) = 0 4434 getuid() = 0 4434 geteuid() = 0 4434 getgid() = 0 4434 getegid() = 0 4434 prctl(PR_GET_DUMPABLE)= 1 4434 stat(/run, {st_mode=S_IFDIR|0755, st_size=820, ...}) = 0 4434 lstat(/run/mount/utab, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0 4434 open(/run/mount/utab, O_RDWR|O_CREAT, 0644) = 3 4434 close(3) = 0 4434 mount(/dev/sda1, /r1, ext4, MS_MGC_VAL|MS_NOATIME, NULL -- (w/ 3.6-rc1) [ 89.868843] mount R running task0 4434 4433 0x0009 [ 89.868847] 880c246b7b68 816c9279 880c246b7aa8 880c246b7fd8 [ 89.868851] 880c246b7fd8 4000 88062720cdb0 880c246862d0 [ 89.868855] 000116c0 880623a863c0 880623a863c0 [ 89.868855] Call Trace: [ 89.868858] [816c9279] ? __schedule+0x299/0x770 [ 89.868860] [816c9279] ? __schedule+0x299/0x770 [ 89.868864] [8114a729] ? ext4_get_group_desc+0x49/0xb0 [ 89.868868] [81161d41] ? ext4_calculate_overhead+0x131/0x3e0 [ 89.868871] [81163a3b] ? ext4_fill_super+0x1a4b/0x28d0 [ 89.868875] [810cc301] ? mount_bdev+0x1a1/0x1e0 [ 89.868877] [81161ff0] ? ext4_calculate_overhead+0x3e0/0x3e0 [ 89.868880] [8115dd00] ? ext4_mount+0x10/0x20 [ 89.868882] [810cc55b] ? mount_fs+0x1b/0xd0 [ 89.868885] [810e57af] ? vfs_kern_mount+0x6f/0x110 [ 89.86] [810e58cf] ? do_kern_mount+0x4f/0x100 [ 89.868890] [810e6dae] ? do_mount+0x2fe/0x8a0 [ 89.868894] [8109c0a3] ? strndup_user+0x53/0x70 [ 89.868896] [810e73e0] ? sys_mount+0x90/0xe0 [ 89.868899] [816cafa1] ? tracesys+0xd4/0xd9 Justin. CC: linux-ext4 Any ideas here (kernel 3.4 and below can mount 60TB ext4 no issues) but 3.5.1 (did not try 3.5) cannot mount the filesystem. Justin. Justin. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/