Re: _proxy_pda still makes linking modules fail
On Tue, 2007-03-13 at 16:57 +0100, Andi Kleen wrote: > On Tue, Mar 13, 2007 at 05:23:52PM +1100, Rusty Russell wrote: > > In particular, it's been put in GCC 4.1 for > > CONFIG_CC_STACKPROTECTOR, which assumes %gs:40 will give the stack > > canary. > > Yes that was always ugly, but I don't know a better way. Well, "%gs:__gcc_stack_protector" would have been better. We could have defined __gcc_stack_protector as an absolute symbol (0x40) at the moment, and made it a real per-cpu var later. > > For the record: the PDA should never have existed, that's what percpu > > vars were supposed to be for. Something went wrong here 8( > > PDA predates per cpu. Indeed, but I should have converted it over back in 2003 (?) when the per-cpu stuff went in 8( > > The ideal solution has always been to use __thread, but no architecture > > has yet managed it (I tried for i386, and it quickly caused unbearable > > I tried it too, but __thread is hopeless for kernel code > > > pain). On x86-64 that uses "%fs" on x86-64, not "%gs" as the kernel > > does, but I might try that if I feel particularly masochistic soon... > > Then swapgs wouldn't work anymore (there is no swapfs) Good point. Thanks, Rusty. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: _proxy_pda still makes linking modules fail
On Tue, 2007-03-13 at 08:31 -0700, Jeremy Fitzhardinge wrote: > Paul Mackerras wrote: > > There is a fundamental problem with using __thread, which is that gcc > > assumes that the addresses of __thread variables are constant within > > one thread, and that therefore it can cache the result of address > > calculations. However, with preempt, threads in the kernel can't rely > > on staying on one cpu, and therefore the addresses of per-cpu > > variables can change. There appears to be no way to tell gcc to drop > > all cached __thread variable address calculations at a given point > > (e.g. when enabling or disabling preemption). That is basically why I > > gave up on using __thread for per-cpu variables on powerpc. [ Thanks for the enlightenment, Paul ] > Doesn't that fall under the general class of "you have to be pinned to a > particular cpu in order to meaningfully use per-cpu variables"? No, it makes assumptions about the *address* of a per-cpu variable not changing, even across barriers. > In principle gcc could CSE the value of smp_processor_id() across a cpu > change in the same way. No, this is why preempt_enable and the like are memory barriers. Rusty. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: _proxy_pda still makes linking modules fail
Jeremy Fitzhardinge writes: > Or do you mean that if you have: > > preempt_disable(); > use_my_percpu++; > preempt_enable(); > // switch cpus > preempt_disable(); > use_my_percpu++; > preempt_enable(); > > then it will still use the old pointer to use_my_percpu? Yes. It can, and sometimes does. There's no way (that I know of) to tell gcc "all my __thread variables might have moved to a different address". > In principle gcc could CSE the value of smp_processor_id() across a cpu > change in the same way. There it's easier to make gcc do what we want, because we can use a barrier or a volatile. The difference is that smp_processor_id() is ultimately the value of something, not the address of something. We can tell gcc "values might have changed" but have no way to say "addresses might have changed". Paul. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: _proxy_pda still makes linking modules fail
On Tue, Mar 13, 2007 at 05:23:52PM +1100, Rusty Russell wrote: > On Tue, 2007-03-13 at 08:59 +1100, Rusty Russell wrote: > > On Mon, 2007-03-12 at 10:48 +0100, Andi Kleen wrote: > > > > Rusty's pda->per_cpu patch will deal with this once and for all; have > > > > > > Not on x86-64. > > > > Indeed. Perhaps it's time I join the modern world and compile a 64-bit > > kernel... > > > > Will prepare patches, > > No, I don't think I will. The PDA concept has gone too far in x86-64 to > be undone. In particular, it's been put in GCC 4.1 for > CONFIG_CC_STACKPROTECTOR, which assumes %gs:40 will give the stack > canary. Yes that was always ugly, but I don't know a better way. > For the record: the PDA should never have existed, that's what percpu > vars were supposed to be for. Something went wrong here 8( PDA predates per cpu. > The ideal solution has always been to use __thread, but no architecture > has yet managed it (I tried for i386, and it quickly caused unbearable I tried it too, but __thread is hopeless for kernel code > pain). On x86-64 that uses "%fs" on x86-64, not "%gs" as the kernel > does, but I might try that if I feel particularly masochistic soon... Then swapgs wouldn't work anymore (there is no swapfs) -Andi > - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: _proxy_pda still makes linking modules fail
Paul Mackerras wrote: > There is a fundamental problem with using __thread, which is that gcc > assumes that the addresses of __thread variables are constant within > one thread, and that therefore it can cache the result of address > calculations. However, with preempt, threads in the kernel can't rely > on staying on one cpu, and therefore the addresses of per-cpu > variables can change. There appears to be no way to tell gcc to drop > all cached __thread variable address calculations at a given point > (e.g. when enabling or disabling preemption). That is basically why I > gave up on using __thread for per-cpu variables on powerpc. > Doesn't that fall under the general class of "you have to be pinned to a particular cpu in order to meaningfully use per-cpu variables"? Or do you mean that if you have: preempt_disable(); use_my_percpu++; preempt_enable(); // switch cpus preempt_disable(); use_my_percpu++; preempt_enable(); then it will still use the old pointer to use_my_percpu? In principle gcc could CSE the value of smp_processor_id() across a cpu change in the same way. J - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: _proxy_pda still makes linking modules fail
Rusty Russell writes: > The ideal solution has always been to use __thread, but no architecture > has yet managed it (I tried for i386, and it quickly caused unbearable > pain). On x86-64 that uses "%fs" on x86-64, not "%gs" as the kernel > does, but I might try that if I feel particularly masochistic soon... There is a fundamental problem with using __thread, which is that gcc assumes that the addresses of __thread variables are constant within one thread, and that therefore it can cache the result of address calculations. However, with preempt, threads in the kernel can't rely on staying on one cpu, and therefore the addresses of per-cpu variables can change. There appears to be no way to tell gcc to drop all cached __thread variable address calculations at a given point (e.g. when enabling or disabling preemption). That is basically why I gave up on using __thread for per-cpu variables on powerpc. We could use __thread for per-task variables quite naturally. There doesn't seem to be much point, though, since we already have a way to do per-task variables - it's called struct task_struct. :-/ Paul. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: _proxy_pda still makes linking modules fail
On Tue, 2007-03-13 at 08:59 +1100, Rusty Russell wrote: > On Mon, 2007-03-12 at 10:48 +0100, Andi Kleen wrote: > > > Rusty's pda->per_cpu patch will deal with this once and for all; have > > > > Not on x86-64. > > Indeed. Perhaps it's time I join the modern world and compile a 64-bit > kernel... > > Will prepare patches, No, I don't think I will. The PDA concept has gone too far in x86-64 to be undone. In particular, it's been put in GCC 4.1 for CONFIG_CC_STACKPROTECTOR, which assumes %gs:40 will give the stack canary. For the record: the PDA should never have existed, that's what percpu vars were supposed to be for. Something went wrong here 8( %gs is best set to the offset of the local cpu's area from the "master" per-cpu area, not set to the local cpu area's address. In the former case, booting with %gs at offset 0 works naturally, in the latter case, hoops need to be jumped through to make it work. See how much nicer the x86 code is post pda->percpu conversion. So, even if we leave the PDA and place the per-cpu area immediately after it, we still can't use "%gs:var" to access a per-cpu variable: we need to do a subtract, so why bother using the segment reg? The ideal solution has always been to use __thread, but no architecture has yet managed it (I tried for i386, and it quickly caused unbearable pain). On x86-64 that uses "%fs" on x86-64, not "%gs" as the kernel does, but I might try that if I feel particularly masochistic soon... In summary, containing the PDA infection to x86-64 is possible, but curing that patient is non-trivial 8) Rusty. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: _proxy_pda still makes linking modules fail
On Mon, 2007-03-12 at 10:48 +0100, Andi Kleen wrote: > > Rusty's pda->per_cpu patch will deal with this once and for all; have > > Not on x86-64. Indeed. Perhaps it's time I join the modern world and compile a 64-bit kernel... Will prepare patches, Rusty. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: _proxy_pda still makes linking modules fail
Dnia 12-03-2007, pon o godzinie 02:19 +0100, Andi Kleen napisał(a):
> Hmm, it probably needs a EXPORT_SYMBOL. The previous change only
> fixed the in kernel build.
>
> Does it work with this patch?
Yes! Thank you.
--
__("< Marcin Kowalczyk
\__/ [EMAIL PROTECTED]
^^ http://qrnik.knm.org.pl/~qrczak/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: _proxy_pda still makes linking modules fail
On Mon, Mar 12, 2007 at 07:45:25AM -0700, Jeremy Fitzhardinge wrote: > Andi Kleen wrote: > >> Rusty's pda->per_cpu patch will deal with this once and for all; have > >> > > > > Not on x86-64. > > > > Have you considered dropping pda in x86-64? Segment based percpu > doesn't really have any disadvantages. Maybe. > > >> you picked it up yet? > >> > > > > Not yet. > > > > There will be interactions with my paravirt+xen patches, so I'm > wondering if I should rebase onto those patches, or should we apply them > later and fix everything up? Applying them later is probably better -Andi - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: _proxy_pda still makes linking modules fail
Andi Kleen wrote: >> Rusty's pda->per_cpu patch will deal with this once and for all; have >> > > Not on x86-64. > Have you considered dropping pda in x86-64? Segment based percpu doesn't really have any disadvantages. >> you picked it up yet? >> > > Not yet. > There will be interactions with my paravirt+xen patches, so I'm wondering if I should rebase onto those patches, or should we apply them later and fix everything up? J - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: _proxy_pda still makes linking modules fail
On Sun, Mar 11, 2007 at 05:25:46PM -0700, Jeremy Fitzhardinge wrote: > Andi Kleen wrote: > > Hmm, it probably needs a EXPORT_SYMBOL. The previous change only > > fixed the in kernel build. > > > > Does it work with this patch? > > > > -Andi > > > > Export _proxy_pda for gcc 4.2 > > > > Gak. It seemed like such a good idea at the time. The problem is that the upcomming 4.2 has a more aggressive optimizer and when there are two "m" (_proxy_pda...) references in the same function it CSEs its address into a register. And that generates code and a reference. > > Rusty's pda->per_cpu patch will deal with this once and for all; have Not on x86-64. > you picked it up yet? Not yet. -Andi - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: _proxy_pda still makes linking modules fail
Andi Kleen wrote: > Hmm, it probably needs a EXPORT_SYMBOL. The previous change only > fixed the in kernel build. > > Does it work with this patch? > > -Andi > > Export _proxy_pda for gcc 4.2 > Gak. It seemed like such a good idea at the time. Rusty's pda->per_cpu patch will deal with this once and for all; have you picked it up yet? J - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: _proxy_pda still makes linking modules fail
Marcin 'Qrczak' Kowalczyk <[EMAIL PROTECTED]> writes: > > I've heard that it now builds with gcc-4.2.0 snapshots. This is strange: > if the problem has been fixed for gcc-4.2.0, why doesn't it work for > gcc-4.1.2? arch/i386/kernel/vmlinux.lds.S does contain _proxy_pda = 0; Hmm, it probably needs a EXPORT_SYMBOL. The previous change only fixed the in kernel build. Does it work with this patch? -Andi Export _proxy_pda for gcc 4.2 The symbol is not actually used, but the compiler unforunately generates a (unused) reference to it. This can happen even in modules. So export it. Signed-off-by: Andi Kleen <[EMAIL PROTECTED]> Index: linux/arch/i386/kernel/i386_ksyms.c === --- linux.orig/arch/i386/kernel/i386_ksyms.c +++ linux/arch/i386/kernel/i386_ksyms.c @@ -28,3 +28,5 @@ EXPORT_SYMBOL(__read_lock_failed); #endif EXPORT_SYMBOL(csum_partial); + +EXPORT_SYMBOL(_proxy_pda); Index: linux/arch/x86_64/kernel/x8664_ksyms.c === --- linux.orig/arch/x86_64/kernel/x8664_ksyms.c +++ linux/arch/x86_64/kernel/x8664_ksyms.c @@ -61,3 +61,4 @@ EXPORT_SYMBOL(empty_zero_page); EXPORT_SYMBOL(init_level4_pgt); EXPORT_SYMBOL(load_gs_index); +EXPORT_SYMBOL(_proxy_pda); - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: _proxy_pda still makes linking modules fail
On Thu, Mar 08, 2007 at 01:57:59AM +0100, Marcin 'Qrczak' Kowalczyk wrote: > This is linux-2.6.20.1 with lots of patches from PLD Linux (I believe > the patches don't affect the issue), compiled with gcc-4.1.2 and > binutils-2.17.50.0.12 on x86. >... Please test a plain 2.6.20 from ftp.kernel.org. If it fails, please send your .config . cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
