Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-20 Thread Roland Dreier
BTW, sorry I didn't get a chance to try some of the other debugging you suggested yet... got busy with other stuff. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-20 Thread Roland Dreier
> I just tried Ingo's patch[1] on a 2.6.25-rc2 kernel with printk timestamps > turned on ... and it booted just fine on my tiger4. The default path > for non-boot cpus is from head.S to start_secondary(), and that > calls cpu_init() pretty quickly. There shouldn't normally[2] be any >

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-20 Thread David Miller
From: Ingo Molnar <[EMAIL PROTECTED]> Date: Wed, 20 Feb 2008 09:12:28 +0100 > > * Tony Luck <[EMAIL PROTECTED]> wrote: > > > > Perhaps what is happening is that cpu0 comes online ... safely skips > > > over the early printk calls. Calls cpu_init() which sets up the > > > resources *it* needs

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-20 Thread Ingo Molnar
* Tony Luck <[EMAIL PROTECTED]> wrote: > > Perhaps what is happening is that cpu0 comes online ... safely skips > > over the early printk calls. Calls cpu_init() which sets up the > > resources *it* needs (ar.k3 points to per-cpu space), and then > > executes sched_init() which marks it safe

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-20 Thread Ingo Molnar
* Tony Luck [EMAIL PROTECTED] wrote: Perhaps what is happening is that cpu0 comes online ... safely skips over the early printk calls. Calls cpu_init() which sets up the resources *it* needs (ar.k3 points to per-cpu space), and then executes sched_init() which marks it safe for all

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-20 Thread David Miller
From: Ingo Molnar [EMAIL PROTECTED] Date: Wed, 20 Feb 2008 09:12:28 +0100 * Tony Luck [EMAIL PROTECTED] wrote: Perhaps what is happening is that cpu0 comes online ... safely skips over the early printk calls. Calls cpu_init() which sets up the resources *it* needs (ar.k3 points

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-20 Thread Roland Dreier
I just tried Ingo's patch[1] on a 2.6.25-rc2 kernel with printk timestamps turned on ... and it booted just fine on my tiger4. The default path for non-boot cpus is from head.S to start_secondary(), and that calls cpu_init() pretty quickly. There shouldn't normally[2] be any printk()

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-20 Thread Roland Dreier
BTW, sorry I didn't get a chance to try some of the other debugging you suggested yet... got busy with other stuff. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-19 Thread Tony Luck
> Perhaps what is happening is that cpu0 comes online ... safely skips > over the early printk calls. Calls cpu_init() which sets up the resources > *it* needs (ar.k3 points to per-cpu space), and then executes > sched_init() which marks it safe for all printk's. Then cpu1 comes > up and does a

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-19 Thread Tony Luck
Perhaps what is happening is that cpu0 comes online ... safely skips over the early printk calls. Calls cpu_init() which sets up the resources *it* needs (ar.k3 points to per-cpu space), and then executes sched_init() which marks it safe for all printk's. Then cpu1 comes up and does a printk

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-14 Thread Tony Luck
> We *ought* to be safe after cpu_init() ... which is called from setup_arch(), > which is several calls before sched_init(). Perhaps what is happening is that cpu0 comes online ... safely skips over the early printk calls. Calls cpu_init() which sets up the resources *it* needs (ar.k3 points to

RE: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-14 Thread Luck, Tony
> I guess sched_init() is too early... it does seem really strange to > me, but I just double checked with Ingo's patch and it does indeed > hang. The slow way to make progress is just to go through > start_kernel() line-by-line and enable cpu_clock() at each stage, and > see where it stops

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-14 Thread Roland Dreier
> > The strange thing is that Ingo's patch to make cpu_clock() a NOP until > > after sched_init() didn't fix things for me... > Very strange. I threw in an output line counter into the printk code() ... > if I > disable the timestamps for the first 30 lines, then everything is good (so

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-14 Thread Tony Luck
On Wed, Feb 13, 2008 at 7:47 PM, Roland Dreier <[EMAIL PROTECTED]> wrote: > The strange thing is that Ingo's patch to make cpu_clock() a NOP until > after sched_init() didn't fix things for me... Very strange. I threw in an output line counter into the printk code() ... if I disable the

RE: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-14 Thread Luck, Tony
> Just curious -- can you reproduce the same problem with > CONFIG_PRINTK_TIME as I'm seeing? Yes I can reproduce this (on latest Linus tree). System dies with no console output ... looks like the boot cpu may have taken a machine check (it isn't responding to my debugger). -Tony -- To

RE: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-14 Thread Luck, Tony
Just curious -- can you reproduce the same problem with CONFIG_PRINTK_TIME as I'm seeing? Yes I can reproduce this (on latest Linus tree). System dies with no console output ... looks like the boot cpu may have taken a machine check (it isn't responding to my debugger). -Tony -- To

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-14 Thread Tony Luck
On Wed, Feb 13, 2008 at 7:47 PM, Roland Dreier [EMAIL PROTECTED] wrote: The strange thing is that Ingo's patch to make cpu_clock() a NOP until after sched_init() didn't fix things for me... Very strange. I threw in an output line counter into the printk code() ... if I disable the timestamps

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-14 Thread Roland Dreier
The strange thing is that Ingo's patch to make cpu_clock() a NOP until after sched_init() didn't fix things for me... Very strange. I threw in an output line counter into the printk code() ... if I disable the timestamps for the first 30 lines, then everything is good (so the

RE: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-14 Thread Luck, Tony
I guess sched_init() is too early... it does seem really strange to me, but I just double checked with Ingo's patch and it does indeed hang. The slow way to make progress is just to go through start_kernel() line-by-line and enable cpu_clock() at each stage, and see where it stops hanging.

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-14 Thread Tony Luck
We *ought* to be safe after cpu_init() ... which is called from setup_arch(), which is several calls before sched_init(). Perhaps what is happening is that cpu0 comes online ... safely skips over the early printk calls. Calls cpu_init() which sets up the resources *it* needs (ar.k3 points to

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-13 Thread Roland Dreier
> I'll take a closer look at what is needed tomorrow. Hi Tony, Just curious -- can you reproduce the same problem with CONFIG_PRINTK_TIME as I'm seeing? If not I'm happy to test anything you want to try. The strange thing is that Ingo's patch to make cpu_clock() a NOP until after sched_init()

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-13 Thread Tony Luck
> That's right. I thought you guys had something that would handle that > early on, but looking at how the trick works in the vmlinux.lds.S ia64 > uses that isn't the case. We try to get things set up pertty early ... but I agree this is fragile. Adding code to printk() to not provide a

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-13 Thread David Miller
From: "Tony Luck" <[EMAIL PROTECTED]> Date: Wed, 13 Feb 2008 16:59:30 -0800 > > It is legal to access per-cpu data as early as you like, > > it just evaluates to the static copy in the per-cpu section > > of the kernel image until the per-cpu areas are setup. > > On ia64 per-cpu variables are

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-13 Thread Tony Luck
> It is legal to access per-cpu data as early as you like, > it just evaluates to the static copy in the per-cpu section > of the kernel image until the per-cpu areas are setup. On ia64 per-cpu variables are mapped into the top 64K of the address space. Accessing them before the resources to

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-13 Thread Roland Dreier
> > so .. how about the patch below? Note that we already had an "early > > bootup" special (the rq->idle check), it's now just made explicit via > > the scheduler_running flag. > > the one below even builds. (untested otherwise) I just tried this... it doesn't work on top of current git

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-13 Thread Ingo Molnar
* David Miller <[EMAIL PROTECTED]> wrote: > I see nothing illegal in what cpu_clock() is doing, that's why I fixed > the sparc64 per-cpu problem I ran into since sparc64 was doing the > wrong thing when booted on a non-zero cpu. okay. I'm just somewhat uneasy about potentially losing the

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-13 Thread Ingo Molnar
* Ingo Molnar <[EMAIL PROTECTED]> wrote: > so .. how about the patch below? Note that we already had an "early > bootup" special (the rq->idle check), it's now just made explicit via > the scheduler_running flag. the one below even builds. (untested otherwise) Ingo --->

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-13 Thread David Miller
From: Ingo Molnar <[EMAIL PROTECTED]> Date: Wed, 13 Feb 2008 13:57:25 +0100 > so .. how about the patch below? Note that we already had an "early > bootup" special (the rq->idle check), it's now just made explicit via > the scheduler_running flag. I don't see what the problem is. It is legal

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-13 Thread Ingo Molnar
* David Miller <[EMAIL PROTECTED]> wrote: > The kernel now derefernces per-cpu variables very early, essentially > in the very first printk() (via printk()'s call to cpu_clock()). > > This bit me on sparc64 because of how I do the per-cpu address > formation. If I booted on a non-zero cpuid

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-13 Thread Ingo Molnar
* David Miller [EMAIL PROTECTED] wrote: The kernel now derefernces per-cpu variables very early, essentially in the very first printk() (via printk()'s call to cpu_clock()). This bit me on sparc64 because of how I do the per-cpu address formation. If I booted on a non-zero cpuid things

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-13 Thread Ingo Molnar
* Ingo Molnar [EMAIL PROTECTED] wrote: so .. how about the patch below? Note that we already had an early bootup special (the rq-idle check), it's now just made explicit via the scheduler_running flag. the one below even builds. (untested otherwise) Ingo --- Subject:

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-13 Thread David Miller
From: Ingo Molnar [EMAIL PROTECTED] Date: Wed, 13 Feb 2008 13:57:25 +0100 so .. how about the patch below? Note that we already had an early bootup special (the rq-idle check), it's now just made explicit via the scheduler_running flag. I don't see what the problem is. It is legal to

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-13 Thread Ingo Molnar
* David Miller [EMAIL PROTECTED] wrote: I see nothing illegal in what cpu_clock() is doing, that's why I fixed the sparc64 per-cpu problem I ran into since sparc64 was doing the wrong thing when booted on a non-zero cpu. okay. I'm just somewhat uneasy about potentially losing the ability

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-13 Thread Roland Dreier
so .. how about the patch below? Note that we already had an early bootup special (the rq-idle check), it's now just made explicit via the scheduler_running flag. the one below even builds. (untested otherwise) I just tried this... it doesn't work on top of current git (same

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-13 Thread Tony Luck
It is legal to access per-cpu data as early as you like, it just evaluates to the static copy in the per-cpu section of the kernel image until the per-cpu areas are setup. On ia64 per-cpu variables are mapped into the top 64K of the address space. Accessing them before the resources to handle

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-13 Thread David Miller
From: Tony Luck [EMAIL PROTECTED] Date: Wed, 13 Feb 2008 16:59:30 -0800 It is legal to access per-cpu data as early as you like, it just evaluates to the static copy in the per-cpu section of the kernel image until the per-cpu areas are setup. On ia64 per-cpu variables are mapped into

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-13 Thread Tony Luck
That's right. I thought you guys had something that would handle that early on, but looking at how the trick works in the vmlinux.lds.S ia64 uses that isn't the case. We try to get things set up pertty early ... but I agree this is fragile. Adding code to printk() to not provide a timestamp

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-13 Thread Roland Dreier
I'll take a closer look at what is needed tomorrow. Hi Tony, Just curious -- can you reproduce the same problem with CONFIG_PRINTK_TIME as I'm seeing? If not I'm happy to test anything you want to try. The strange thing is that Ingo's patch to make cpu_clock() a NOP until after sched_init()

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-12 Thread David Miller
From: Roland Dreier <[EMAIL PROTECTED]> Date: Tue, 12 Feb 2008 22:24:05 -0800 > I'm seeing a strange hang with current git (head 96b5a46e) on an ia64 > box -- an Intel SDV with 2 dual core hyperthreaded Itanium 2 CPUs (so > 8 logical CPUs to the kernel). It hangs without printing anything >

Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-12 Thread Roland Dreier
I'm seeing a strange hang with current git (head 96b5a46e) on an ia64 box -- an Intel SDV with 2 dual core hyperthreaded Itanium 2 CPUs (so 8 logical CPUs to the kernel). It hangs without printing anything ("Uncompressing Linux... done" from ELILO is the last thing I see) if I have

Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-12 Thread Roland Dreier
I'm seeing a strange hang with current git (head 96b5a46e) on an ia64 box -- an Intel SDV with 2 dual core hyperthreaded Itanium 2 CPUs (so 8 logical CPUs to the kernel). It hangs without printing anything (Uncompressing Linux... done from ELILO is the last thing I see) if I have

Re: Strange hang on ia64 with CONFIG_PRINTK_TIME=y

2008-02-12 Thread David Miller
From: Roland Dreier [EMAIL PROTECTED] Date: Tue, 12 Feb 2008 22:24:05 -0800 I'm seeing a strange hang with current git (head 96b5a46e) on an ia64 box -- an Intel SDV with 2 dual core hyperthreaded Itanium 2 CPUs (so 8 logical CPUs to the kernel). It hangs without printing anything