Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
Martin Bligh wrote: > Christoph Lameter wrote: >> On Fri, 16 Mar 2007, Martin Bligh wrote: >> >>> You have to do some sort of lookup anyway, and Andy seemed to have them >>> all folded into one. >> >> What lookup would you need to do? On x86_64 even the TLB use is hidden >> by the existing 2M entries for 1-1 mappings. >> >>> Or are you trying to avoid this by going to back to the crud we had >>> in 2.4 where we pretend mem_map is one big array, indexed by pfn with >>> huge sparsely mapped holes in it? >> >> Yes that the advanced way of doing it rather than adding useless >> custom lookups. > > For starters, you can't do that sparse a mapping on a 32 bit system. > I'll let Andy explain the rest of it. > >>> Would be nice to work out (and document somewhere) what the pros and >>> cons of virtual memmap vs sparsemem were - ISTR one of the arguments >>> was extremely sparsely layed out machines, and you needed sparsemem >>> for that. But right now we have 3 solutions, which is not a good >>> situation. >> >> Please read my posts to linux-mm on that subject. We discussed it last >> year in detail and the agreement was that the sparsemem crud needs to >> be taken out. Kame-san posted patches to do that. > > "the agreement"? So Andy agreed to taking it out? Or you and Kame did? The discussions centred around some patches from Kame which introduced a SPARSMEM sub-model with a virtual memory map. That was a supprisingly clean change which if followed through to its logical conclusion would remove a significant chunk of architecture specific vmemmap implementation from ia64, and (as I understand it) was likely to allow the same to be reused in s390x as well. SPARSEMEM would still have its useful modes for smaller memory systems. -apw - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
> And those machines are basically identical to perfectly regular i386 > platforms. For modern (2001+) i386 platforms sure. The problem is the old and the weird. > > So the whole argument that it would "diverge" is total crap. It obviously > won't diverge, simply because the support for old setups is needed on > x86-64 *regardless* of whether 32-bit support exists on the same platform > or not. There is still some unneeded old platform code on x86-64 that will be removed (although I'm slowly discovering that some like the irq 0 hacks are still needed :/). i386 support goes much farther back than the earlier x86-64 code supported, not even talking about complications like the weird and fragile subarchs i386 has. And x86-64 since it supports only modern systems is easier to hack because it's much easier to regression test and only needs to care about reasonably modern hardware. That is for example it has new smpboot and i386 doesn't and has extended per CPU IRQ vectors and i386 doesn't and has new machine check code and i386 doesn't etc.etc. It was basically a new start in many areas. -Andi - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
And those machines are basically identical to perfectly regular i386 platforms. For modern (2001+) i386 platforms sure. The problem is the old and the weird. So the whole argument that it would diverge is total crap. It obviously won't diverge, simply because the support for old setups is needed on x86-64 *regardless* of whether 32-bit support exists on the same platform or not. There is still some unneeded old platform code on x86-64 that will be removed (although I'm slowly discovering that some like the irq 0 hacks are still needed :/). i386 support goes much farther back than the earlier x86-64 code supported, not even talking about complications like the weird and fragile subarchs i386 has. And x86-64 since it supports only modern systems is easier to hack because it's much easier to regression test and only needs to care about reasonably modern hardware. That is for example it has new smpboot and i386 doesn't and has extended per CPU IRQ vectors and i386 doesn't and has new machine check code and i386 doesn't etc.etc. It was basically a new start in many areas. -Andi - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
Martin Bligh wrote: Christoph Lameter wrote: On Fri, 16 Mar 2007, Martin Bligh wrote: You have to do some sort of lookup anyway, and Andy seemed to have them all folded into one. What lookup would you need to do? On x86_64 even the TLB use is hidden by the existing 2M entries for 1-1 mappings. Or are you trying to avoid this by going to back to the crud we had in 2.4 where we pretend mem_map is one big array, indexed by pfn with huge sparsely mapped holes in it? Yes that the advanced way of doing it rather than adding useless custom lookups. For starters, you can't do that sparse a mapping on a 32 bit system. I'll let Andy explain the rest of it. Would be nice to work out (and document somewhere) what the pros and cons of virtual memmap vs sparsemem were - ISTR one of the arguments was extremely sparsely layed out machines, and you needed sparsemem for that. But right now we have 3 solutions, which is not a good situation. Please read my posts to linux-mm on that subject. We discussed it last year in detail and the agreement was that the sparsemem crud needs to be taken out. Kame-san posted patches to do that. the agreement? So Andy agreed to taking it out? Or you and Kame did? The discussions centred around some patches from Kame which introduced a SPARSMEM sub-model with a virtual memory map. That was a supprisingly clean change which if followed through to its logical conclusion would remove a significant chunk of architecture specific vmemmap implementation from ia64, and (as I understand it) was likely to allow the same to be reused in s390x as well. SPARSEMEM would still have its useful modes for smaller memory systems. -apw - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Fri, 16 Mar 2007, Andi Kleen wrote: > > In the future it is likely that x86_64 will significantly deviate from > > It already is in some cases. And I agree more will happen. This is a *totally* bogus and idiotic argument. x86-64 will get new capabilities, BUT IT WILL CONTINUE TO SUPPORT OLD x86-64 machines. And those machines are basically identical to perfectly regular i386 platforms. So the whole argument that it would "diverge" is total crap. It obviously won't diverge, simply because the support for old setups is needed on x86-64 *regardless* of whether 32-bit support exists on the same platform or not. There's a huge difference between divergence and "more capabilities". Linus - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Fri, 16 Mar 2007, Andi Kleen wrote: In the future it is likely that x86_64 will significantly deviate from It already is in some cases. And I agree more will happen. This is a *totally* bogus and idiotic argument. x86-64 will get new capabilities, BUT IT WILL CONTINUE TO SUPPORT OLD x86-64 machines. And those machines are basically identical to perfectly regular i386 platforms. So the whole argument that it would diverge is total crap. It obviously won't diverge, simply because the support for old setups is needed on x86-64 *regardless* of whether 32-bit support exists on the same platform or not. There's a huge difference between divergence and more capabilities. Linus - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Fri, 16 Mar 2007, Christoph Lameter wrote: > > Yes he has already explained it and I am well aware of the difficulties > on 32 bit. -> linux-mm archives. Stop pointing to archives. If you cannot give a http pointer to a specific thread, don't bother with the "please real the list" thing AT ALL. And I'm sorry, "we decided this on linux-mm" doesn't cut it as an explanation _or_ as a "it's already been decided". Many relevant people simply aren't on that mailing list. Linus - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Fri, 16 Mar 2007, Martin Bligh wrote: > For starters, you can't do that sparse a mapping on a 32 bit system. > I'll let Andy explain the rest of it. Yes he has already explained it and I am well aware of the difficulties on 32 bit. -> linux-mm archives. > "the agreement"? So Andy agreed to taking it out? Or you and Kame did? Yes Andy and lots of others. Big discussion that you seem to want me to repeat here. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
Christoph Lameter wrote: On Fri, 16 Mar 2007, Martin Bligh wrote: You have to do some sort of lookup anyway, and Andy seemed to have them all folded into one. What lookup would you need to do? On x86_64 even the TLB use is hidden by the existing 2M entries for 1-1 mappings. Or are you trying to avoid this by going to back to the crud we had in 2.4 where we pretend mem_map is one big array, indexed by pfn with huge sparsely mapped holes in it? Yes that the advanced way of doing it rather than adding useless custom lookups. For starters, you can't do that sparse a mapping on a 32 bit system. I'll let Andy explain the rest of it. Would be nice to work out (and document somewhere) what the pros and cons of virtual memmap vs sparsemem were - ISTR one of the arguments was extremely sparsely layed out machines, and you needed sparsemem for that. But right now we have 3 solutions, which is not a good situation. Please read my posts to linux-mm on that subject. We discussed it last year in detail and the agreement was that the sparsemem crud needs to be taken out. Kame-san posted patches to do that. "the agreement"? So Andy agreed to taking it out? Or you and Kame did? M. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Fri, 2007-03-16 at 13:15 -0700, Christoph Lameter wrote: > On Fri, 16 Mar 2007, Andi Kleen wrote: > > > x86_64 is going to acquire more functionality that will not be available > > > for i386. We plan f.e. to add virtual memmap support for x86_64. Virtual > > > > What advantage would that have over the current setup? > > We already should handle holes between nodes reasonably efficiently > > and with nonlinear memory even holes inside nodes shouldn't be a problem. > > It is primarily a performance improvement since the sparsemem table > lookups would no longer be necessary and it also streamlines other > frequent cacheline uses. These page -> page_struct and vice versa > operations are key to the performance of various subsystem among them > the slab allocator. Hi Christoph, Yeah, those are horribly common operations. But, have we actually quantified how sparsemem hurts here? I know that it look on the surface like those lookups should kill you but, as far as I can tell, we've never been able to show that they actually do. We were worried on the NUMAQ that we would see some performance regressions, but it somehow slightly outperformed discontigmem. Do you have any hard data that shows vmemmap killing sparsemem on some of your workloads? I'd love to try and fix some of the issues if we can dig them up. One of the reasons I really like sparsemem is that ports to new architectures are something like 100 lines of code, including all of the #defines. It's really ridiculously easy to do. One of the things that I really worry about for vmemmap implementations is how complicated they get to implement. The fact that we can't easily do it for both i386 and x86_64 at the same time speaks to this a bit. -- Dave - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Fri, 16 Mar 2007, David Miller wrote: > From: Christoph Lameter <[EMAIL PROTECTED]> > Date: Fri, 16 Mar 2007 13:48:58 -0700 (PDT) > > > Please read my posts to linux-mm on that subject. We discussed it last > > year in detail and the agreement was that the sparsemem crud needs to be > > taken out. Kame-san posted patches to do that. > > Please don't do that, sparsemem works very well on sparc64 and I > like the flexibility it gives me. I am not sure what flexibility you are talking about? The modification are to make sparsemem support virtual memmap as one option. See Kame-san's posts. There is no regression here its just cutting out the overhead of sparsmem from within. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
From: Christoph Lameter <[EMAIL PROTECTED]> Date: Fri, 16 Mar 2007 13:56:13 -0700 (PDT) > On Fri, 16 Mar 2007, David Miller wrote: > > > From: Christoph Lameter <[EMAIL PROTECTED]> > > Date: Fri, 16 Mar 2007 13:48:58 -0700 (PDT) > > > > > Please read my posts to linux-mm on that subject. We discussed it last > > > year in detail and the agreement was that the sparsemem crud needs to be > > > taken out. Kame-san posted patches to do that. > > > > Please don't do that, sparsemem works very well on sparc64 and I > > like the flexibility it gives me. > > I am not sure what flexibility you are talking about? The modification are > to make sparsemem support virtual memmap as one option. See Kame-san's > posts. There is no regression here its just cutting out the overhead of > sparsmem from within. I would really appreciate a posting that explains what exactly is going on being sent to linux-arch, this is the first time I myself am even aware of this idea and work. Thanks. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Fri, 16 Mar 2007, David Miller wrote: > From: Christoph Lameter <[EMAIL PROTECTED]> > Date: Fri, 16 Mar 2007 13:52:18 -0700 (PDT) > > > Virtual mmap allows holes in the same way as page tables do. > > I don't want to take expensive TLB misses to lookup a page. Ummm. You are missing key details please look at linux-mm. We will use huge tlb entries for the mmap and its optional. > TLB misses are extremely expensive on some platforms, that's why > I try to make all the critical structures go through the 4MB > locked TLB entry for the kernel image, or failing that the > 4MB linear mappings at PAGE_OFFSET on sparc64. Exactly. That is why 16M TLB entries speed up the kernel by 4-5% using vmmemmap on ia64. Basically we end up with one TLB per node for the vmemmap which always stays in the cache. No additional lookups. Without vmemmap we still need a 16M TLB for the 1-1 kernel mapping. This jsut cuts out the crud from sparsemem. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
From: Christoph Lameter <[EMAIL PROTECTED]> Date: Fri, 16 Mar 2007 13:52:18 -0700 (PDT) > Virtual mmap allows holes in the same way as page tables do. I don't want to take expensive TLB misses to lookup a page. Don't force a virtual mapping solution down my throat if that is not what I believe as a platform maintainer works best for me. TLB misses are extremely expensive on some platforms, that's why I try to make all the critical structures go through the 4MB locked TLB entry for the kernel image, or failing that the 4MB linear mappings at PAGE_OFFSET on sparc64. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Fri, 16 Mar 2007, David Miller wrote: > > It is primarily a performance improvement since the sparsemem table > > lookups would no longer be necessary and it also streamlines other > > frequent cacheline uses. These page -> page_struct and vice versa > > operations are key to the performance of various subsystem among them > > the slab allocator. > > If you set the bit range small enough and don't use sparsemem-extreme, > the cost is extremely low considering the flexibility you obtain. What flexibility? > There are always going to be holes on large systems, there isn't > really a way to avoid this given how addressing is done on those > machines. Virtual mmap allows holes in the same way as page tables do. These structures were designed for sparseness. I really do not think we should rediscuss the material here that was discussed on linux-mm. The outcome of that was that we will introduce virtual memmap to reduce sparsemem overhead and to increase flexibility. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
From: Christoph Lameter <[EMAIL PROTECTED]> Date: Fri, 16 Mar 2007 13:48:58 -0700 (PDT) > Please read my posts to linux-mm on that subject. We discussed it last > year in detail and the agreement was that the sparsemem crud needs to be > taken out. Kame-san posted patches to do that. Please don't do that, sparsemem works very well on sparc64 and I like the flexibility it gives me. Or are you just proposing to stop using it on x86_64? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Fri, 16 Mar 2007, Martin Bligh wrote: > You have to do some sort of lookup anyway, and Andy seemed to have them > all folded into one. What lookup would you need to do? On x86_64 even the TLB use is hidden by the existing 2M entries for 1-1 mappings. > Or are you trying to avoid this by going to back to the crud we had > in 2.4 where we pretend mem_map is one big array, indexed by pfn with > huge sparsely mapped holes in it? Yes that the advanced way of doing it rather than adding useless custom lookups. > Would be nice to work out (and document somewhere) what the pros and > cons of virtual memmap vs sparsemem were - ISTR one of the arguments > was extremely sparsely layed out machines, and you needed sparsemem > for that. But right now we have 3 solutions, which is not a good > situation. Please read my posts to linux-mm on that subject. We discussed it last year in detail and the agreement was that the sparsemem crud needs to be taken out. Kame-san posted patches to do that. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
From: Christoph Lameter <[EMAIL PROTECTED]> Date: Fri, 16 Mar 2007 13:15:38 -0700 (PDT) > On Fri, 16 Mar 2007, Andi Kleen wrote: > > > > x86_64 is going to acquire more functionality that will not be available > > > for i386. We plan f.e. to add virtual memmap support for x86_64. Virtual > > > > What advantage would that have over the current setup? > > We already should handle holes between nodes reasonably efficiently > > and with nonlinear memory even holes inside nodes shouldn't be a problem. > > It is primarily a performance improvement since the sparsemem table > lookups would no longer be necessary and it also streamlines other > frequent cacheline uses. These page -> page_struct and vice versa > operations are key to the performance of various subsystem among them > the slab allocator. If you set the bit range small enough and don't use sparsemem-extreme, the cost is extremely low considering the flexibility you obtain. There are always going to be holes on large systems, there isn't really a way to avoid this given how addressing is done on those machines. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
Christoph Lameter wrote: On Fri, 16 Mar 2007, Andi Kleen wrote: x86_64 is going to acquire more functionality that will not be available for i386. We plan f.e. to add virtual memmap support for x86_64. Virtual What advantage would that have over the current setup? We already should handle holes between nodes reasonably efficiently and with nonlinear memory even holes inside nodes shouldn't be a problem. It is primarily a performance improvement since the sparsemem table lookups would no longer be necessary and it also streamlines other frequent cacheline uses. These page -> page_struct and vice versa operations are key to the performance of various subsystem among them the slab allocator. cc: apw You have to do some sort of lookup anyway, and Andy seemed to have them all folded into one. Or are you trying to avoid this by going to back to the crud we had in 2.4 where we pretend mem_map is one big array, indexed by pfn with huge sparsely mapped holes in it? Would be nice to work out (and document somewhere) what the pros and cons of virtual memmap vs sparsemem were - ISTR one of the arguments was extremely sparsely layed out machines, and you needed sparsemem for that. But right now we have 3 solutions, which is not a good situation. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Fri, 16 Mar 2007, Andi Kleen wrote: > > > x86_64 is going to acquire more functionality that will not be available > > for i386. We plan f.e. to add virtual memmap support for x86_64. Virtual > > What advantage would that have over the current setup? > We already should handle holes between nodes reasonably efficiently > and with nonlinear memory even holes inside nodes shouldn't be a problem. It is primarily a performance improvement since the sparsemem table lookups would no longer be necessary and it also streamlines other frequent cacheline uses. These page -> page_struct and vice versa operations are key to the performance of various subsystem among them the slab allocator. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
> In the future it is likely that x86_64 will significantly deviate from It already is in some cases. And I agree more will happen. > i386. i386 is going to be gradually abandoned because it does not support > the ever larger memory sizes and be mainly used for embedded devices. The desktop/server user base is quite conservative so that will take a long time, but gradually probably. > x86_64 is going to acquire more functionality that will not be available > for i386. We plan f.e. to add virtual memmap support for x86_64. Virtual What advantage would that have over the current setup? We already should handle holes between nodes reasonably efficiently and with nonlinear memory even holes inside nodes shouldn't be a problem. > memmap support may require a large chunk of virtual memory space that is > not available on i386. Its not good to have to deal with i386 issues when > doing x86_64 arch development. If the great merged architecture ever happens this would be needed anyways. Just basic 32bit support isn't too difficult anyways -- the problem are more the old bugs. -Andi - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
In the future it is likely that x86_64 will significantly deviate from It already is in some cases. And I agree more will happen. i386. i386 is going to be gradually abandoned because it does not support the ever larger memory sizes and be mainly used for embedded devices. The desktop/server user base is quite conservative so that will take a long time, but gradually probably. x86_64 is going to acquire more functionality that will not be available for i386. We plan f.e. to add virtual memmap support for x86_64. Virtual What advantage would that have over the current setup? We already should handle holes between nodes reasonably efficiently and with nonlinear memory even holes inside nodes shouldn't be a problem. memmap support may require a large chunk of virtual memory space that is not available on i386. Its not good to have to deal with i386 issues when doing x86_64 arch development. If the great merged architecture ever happens this would be needed anyways. Just basic 32bit support isn't too difficult anyways -- the problem are more the old bugs. -Andi - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Fri, 16 Mar 2007, Andi Kleen wrote: x86_64 is going to acquire more functionality that will not be available for i386. We plan f.e. to add virtual memmap support for x86_64. Virtual What advantage would that have over the current setup? We already should handle holes between nodes reasonably efficiently and with nonlinear memory even holes inside nodes shouldn't be a problem. It is primarily a performance improvement since the sparsemem table lookups would no longer be necessary and it also streamlines other frequent cacheline uses. These page - page_struct and vice versa operations are key to the performance of various subsystem among them the slab allocator. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
Christoph Lameter wrote: On Fri, 16 Mar 2007, Andi Kleen wrote: x86_64 is going to acquire more functionality that will not be available for i386. We plan f.e. to add virtual memmap support for x86_64. Virtual What advantage would that have over the current setup? We already should handle holes between nodes reasonably efficiently and with nonlinear memory even holes inside nodes shouldn't be a problem. It is primarily a performance improvement since the sparsemem table lookups would no longer be necessary and it also streamlines other frequent cacheline uses. These page - page_struct and vice versa operations are key to the performance of various subsystem among them the slab allocator. cc: apw You have to do some sort of lookup anyway, and Andy seemed to have them all folded into one. Or are you trying to avoid this by going to back to the crud we had in 2.4 where we pretend mem_map is one big array, indexed by pfn with huge sparsely mapped holes in it? Would be nice to work out (and document somewhere) what the pros and cons of virtual memmap vs sparsemem were - ISTR one of the arguments was extremely sparsely layed out machines, and you needed sparsemem for that. But right now we have 3 solutions, which is not a good situation. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
From: Christoph Lameter [EMAIL PROTECTED] Date: Fri, 16 Mar 2007 13:15:38 -0700 (PDT) On Fri, 16 Mar 2007, Andi Kleen wrote: x86_64 is going to acquire more functionality that will not be available for i386. We plan f.e. to add virtual memmap support for x86_64. Virtual What advantage would that have over the current setup? We already should handle holes between nodes reasonably efficiently and with nonlinear memory even holes inside nodes shouldn't be a problem. It is primarily a performance improvement since the sparsemem table lookups would no longer be necessary and it also streamlines other frequent cacheline uses. These page - page_struct and vice versa operations are key to the performance of various subsystem among them the slab allocator. If you set the bit range small enough and don't use sparsemem-extreme, the cost is extremely low considering the flexibility you obtain. There are always going to be holes on large systems, there isn't really a way to avoid this given how addressing is done on those machines. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Fri, 16 Mar 2007, Martin Bligh wrote: You have to do some sort of lookup anyway, and Andy seemed to have them all folded into one. What lookup would you need to do? On x86_64 even the TLB use is hidden by the existing 2M entries for 1-1 mappings. Or are you trying to avoid this by going to back to the crud we had in 2.4 where we pretend mem_map is one big array, indexed by pfn with huge sparsely mapped holes in it? Yes that the advanced way of doing it rather than adding useless custom lookups. Would be nice to work out (and document somewhere) what the pros and cons of virtual memmap vs sparsemem were - ISTR one of the arguments was extremely sparsely layed out machines, and you needed sparsemem for that. But right now we have 3 solutions, which is not a good situation. Please read my posts to linux-mm on that subject. We discussed it last year in detail and the agreement was that the sparsemem crud needs to be taken out. Kame-san posted patches to do that. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Fri, 16 Mar 2007, David Miller wrote: It is primarily a performance improvement since the sparsemem table lookups would no longer be necessary and it also streamlines other frequent cacheline uses. These page - page_struct and vice versa operations are key to the performance of various subsystem among them the slab allocator. If you set the bit range small enough and don't use sparsemem-extreme, the cost is extremely low considering the flexibility you obtain. What flexibility? There are always going to be holes on large systems, there isn't really a way to avoid this given how addressing is done on those machines. Virtual mmap allows holes in the same way as page tables do. These structures were designed for sparseness. I really do not think we should rediscuss the material here that was discussed on linux-mm. The outcome of that was that we will introduce virtual memmap to reduce sparsemem overhead and to increase flexibility. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
From: Christoph Lameter [EMAIL PROTECTED] Date: Fri, 16 Mar 2007 13:48:58 -0700 (PDT) Please read my posts to linux-mm on that subject. We discussed it last year in detail and the agreement was that the sparsemem crud needs to be taken out. Kame-san posted patches to do that. Please don't do that, sparsemem works very well on sparc64 and I like the flexibility it gives me. Or are you just proposing to stop using it on x86_64? - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
From: Christoph Lameter [EMAIL PROTECTED] Date: Fri, 16 Mar 2007 13:52:18 -0700 (PDT) Virtual mmap allows holes in the same way as page tables do. I don't want to take expensive TLB misses to lookup a page. Don't force a virtual mapping solution down my throat if that is not what I believe as a platform maintainer works best for me. TLB misses are extremely expensive on some platforms, that's why I try to make all the critical structures go through the 4MB locked TLB entry for the kernel image, or failing that the 4MB linear mappings at PAGE_OFFSET on sparc64. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Fri, 16 Mar 2007, David Miller wrote: From: Christoph Lameter [EMAIL PROTECTED] Date: Fri, 16 Mar 2007 13:48:58 -0700 (PDT) Please read my posts to linux-mm on that subject. We discussed it last year in detail and the agreement was that the sparsemem crud needs to be taken out. Kame-san posted patches to do that. Please don't do that, sparsemem works very well on sparc64 and I like the flexibility it gives me. I am not sure what flexibility you are talking about? The modification are to make sparsemem support virtual memmap as one option. See Kame-san's posts. There is no regression here its just cutting out the overhead of sparsmem from within. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
From: Christoph Lameter [EMAIL PROTECTED] Date: Fri, 16 Mar 2007 13:56:13 -0700 (PDT) On Fri, 16 Mar 2007, David Miller wrote: From: Christoph Lameter [EMAIL PROTECTED] Date: Fri, 16 Mar 2007 13:48:58 -0700 (PDT) Please read my posts to linux-mm on that subject. We discussed it last year in detail and the agreement was that the sparsemem crud needs to be taken out. Kame-san posted patches to do that. Please don't do that, sparsemem works very well on sparc64 and I like the flexibility it gives me. I am not sure what flexibility you are talking about? The modification are to make sparsemem support virtual memmap as one option. See Kame-san's posts. There is no regression here its just cutting out the overhead of sparsmem from within. I would really appreciate a posting that explains what exactly is going on being sent to linux-arch, this is the first time I myself am even aware of this idea and work. Thanks. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Fri, 16 Mar 2007, David Miller wrote: From: Christoph Lameter [EMAIL PROTECTED] Date: Fri, 16 Mar 2007 13:52:18 -0700 (PDT) Virtual mmap allows holes in the same way as page tables do. I don't want to take expensive TLB misses to lookup a page. Ummm. You are missing key details please look at linux-mm. We will use huge tlb entries for the mmap and its optional. TLB misses are extremely expensive on some platforms, that's why I try to make all the critical structures go through the 4MB locked TLB entry for the kernel image, or failing that the 4MB linear mappings at PAGE_OFFSET on sparc64. Exactly. That is why 16M TLB entries speed up the kernel by 4-5% using vmmemmap on ia64. Basically we end up with one TLB per node for the vmemmap which always stays in the cache. No additional lookups. Without vmemmap we still need a 16M TLB for the 1-1 kernel mapping. This jsut cuts out the crud from sparsemem. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
Christoph Lameter wrote: On Fri, 16 Mar 2007, Martin Bligh wrote: You have to do some sort of lookup anyway, and Andy seemed to have them all folded into one. What lookup would you need to do? On x86_64 even the TLB use is hidden by the existing 2M entries for 1-1 mappings. Or are you trying to avoid this by going to back to the crud we had in 2.4 where we pretend mem_map is one big array, indexed by pfn with huge sparsely mapped holes in it? Yes that the advanced way of doing it rather than adding useless custom lookups. For starters, you can't do that sparse a mapping on a 32 bit system. I'll let Andy explain the rest of it. Would be nice to work out (and document somewhere) what the pros and cons of virtual memmap vs sparsemem were - ISTR one of the arguments was extremely sparsely layed out machines, and you needed sparsemem for that. But right now we have 3 solutions, which is not a good situation. Please read my posts to linux-mm on that subject. We discussed it last year in detail and the agreement was that the sparsemem crud needs to be taken out. Kame-san posted patches to do that. the agreement? So Andy agreed to taking it out? Or you and Kame did? M. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Fri, 2007-03-16 at 13:15 -0700, Christoph Lameter wrote: On Fri, 16 Mar 2007, Andi Kleen wrote: x86_64 is going to acquire more functionality that will not be available for i386. We plan f.e. to add virtual memmap support for x86_64. Virtual What advantage would that have over the current setup? We already should handle holes between nodes reasonably efficiently and with nonlinear memory even holes inside nodes shouldn't be a problem. It is primarily a performance improvement since the sparsemem table lookups would no longer be necessary and it also streamlines other frequent cacheline uses. These page - page_struct and vice versa operations are key to the performance of various subsystem among them the slab allocator. Hi Christoph, Yeah, those are horribly common operations. But, have we actually quantified how sparsemem hurts here? I know that it look on the surface like those lookups should kill you but, as far as I can tell, we've never been able to show that they actually do. We were worried on the NUMAQ that we would see some performance regressions, but it somehow slightly outperformed discontigmem. Do you have any hard data that shows vmemmap killing sparsemem on some of your workloads? I'd love to try and fix some of the issues if we can dig them up. One of the reasons I really like sparsemem is that ports to new architectures are something like 100 lines of code, including all of the #defines. It's really ridiculously easy to do. One of the things that I really worry about for vmemmap implementations is how complicated they get to implement. The fact that we can't easily do it for both i386 and x86_64 at the same time speaks to this a bit. -- Dave - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Fri, 16 Mar 2007, Martin Bligh wrote: For starters, you can't do that sparse a mapping on a 32 bit system. I'll let Andy explain the rest of it. Yes he has already explained it and I am well aware of the difficulties on 32 bit. - linux-mm archives. the agreement? So Andy agreed to taking it out? Or you and Kame did? Yes Andy and lots of others. Big discussion that you seem to want me to repeat here. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Fri, 16 Mar 2007, Christoph Lameter wrote: Yes he has already explained it and I am well aware of the difficulties on 32 bit. - linux-mm archives. Stop pointing to archives. If you cannot give a http pointer to a specific thread, don't bother with the please real the list thing AT ALL. And I'm sorry, we decided this on linux-mm doesn't cut it as an explanation _or_ as a it's already been decided. Many relevant people simply aren't on that mailing list. Linus - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Thu, 15 Mar 2007, Steven Rostedt wrote: > On Thu, 2007-03-15 at 17:06 +0100, Andi Kleen wrote: > > > Well I just see a lot of pain from these patches but I doubt > > they will avoid any bugs. If people don't compile test both > > archs they will always likely break on another. There are lots > > of subtle dependencies that are not expressed in the pathname > > even after this intrusive operation (e.g. in the includes). > > > > That's just how it is. > > Or that's just how you see it. In the future it is likely that x86_64 will significantly deviate from i386. i386 is going to be gradually abandoned because it does not support the ever larger memory sizes and be mainly used for embedded devices. x86_64 is going to acquire more functionality that will not be available for i386. We plan f.e. to add virtual memmap support for x86_64. Virtual memmap support may require a large chunk of virtual memory space that is not available on i386. Its not good to have to deal with i386 issues when doing x86_64 arch development. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Wed, 2007-03-14 at 01:08 -0400, Steven Rostedt wrote: > [Hopefully fixed email client to make it to the list this time] > [This series has changed by using git-diff -M] > Seems appropriate, but I really don't care what it's called. One thing about > this name, is that typing arch/x86 doesn't tab complete x86_64 anymore. > But if you can think of something better, I'd be happy to apply it. > sorry for being so late, but about what it could be called, well, what about common_x86 or common/x86 or something? > > -- Steve > > PS. Sorry for the spam. I need to figure out how to tame quilt mail! > > > -- > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to [EMAIL PROTECTED] > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ > - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Mar 15 2007 08:59, Linus Torvalds wrote: >On Thu, 15 Mar 2007, Martin Bligh wrote: >> >> Can't we move the shared files into a new shared arch/ subdirectory >> (ia32_64 or whatever), and have them included from both places? > >That's *exactly* what the patches do (except it's called "arch/x86", which >is clearly the best option - please don't use "ia" _anywhere_ except for >"ia64", since that's the only architecture that is really "intel >architecture"). IA32 = Intel/AMD 32 :) Jan -- - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
> You could do both. Have the x86 directory that Linus suggests for shared > files, then have the build system generate the symlinks for you. Symlinks are usually a bad idea because they tend to not work with objdirs. We did that in 2.4. -Andi - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Thu, 2007-03-15 at 18:01 +0100, Andi Kleen wrote: > > Oops, sorry, you did say "in the includes". Yeah, that holds the same > > crap that I'm talking about. > > It's a simple and obvious solution that does exactly what it is > supposed to do. Why do you call it crap? Yes, it's a simple solution. The reason I called it crap was because it does what the other hacks do. It hides from the developer a dependency that can create bugs in the other arch, and is not IMNSHO clean. When one works on one arch, you should not need to be worried about breaking other archs when you stay within the arch/ and asm- directories. -- Steve - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
> Oops, sorry, you did say "in the includes". Yeah, that holds the same > crap that I'm talking about. It's a simple and obvious solution that does exactly what it is supposed to do. Why do you call it crap? -Andi - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Thu, 2007-03-15 at 12:47 -0400, Steven Rostedt wrote: > On Thu, 2007-03-15 at 17:06 +0100, Andi Kleen wrote: > > > Well I just see a lot of pain from these patches but I doubt > > they will avoid any bugs. If people don't compile test both > > archs they will always likely break on another. There are lots > > of subtle dependencies that are not expressed in the pathname > > even after this intrusive operation (e.g. in the includes). [...] > With the proposed patch set, what can break i386 by modifying something > in arch/x86_64, or what can break x86_64 by modifying something in > arch/i386? (not counting the unfinished pci shared code). Oops, sorry, you did say "in the includes". Yeah, that holds the same crap that I'm talking about. $ cat include/asm-i386/stacktrace.h #include Which isn't something that I would like to stay either. So it's just something else that needs to be fixed. Not something that will circumvent what the current patch set is trying to do. -- Steve - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
Ingo Molnar wrote: * Linus Torvalds <[EMAIL PROTECTED]> wrote: symbolic links perhaps? In that case i'd also introduce a common naming scheme: x86_early_printk.c - to make sure we know it right away that those files are bi-arch. Hey, I know! This is a radical idea, but what if we put the name at the head of the file, and called it arch/x86/kernel/early_printk.c instead? Then we could teach each of i386 and x86-64 to include it from that area, and we could put other shared files under the same directory hierarchy so that it would be easy to see which ones are shared? that is nice too, but it has some disadvantages as well in practice. For example i often want to see 'everything' that belongs to an arch in just one subdirectory. That way one can grep it for example, instead of having to grep two separate places. Symlinks would be fine for that, but an explicit split not really i think - unless we can get some really significant chunk of code into that hierarchy, so that it makes functional /sense/ to look at it in isolation. with the prefix suggestion we can keep these 'shared' files merged in a single, main functional tree (x86_64), but still have them marked in the VFS as being shared. But ... either way is fine to me - no strong feelings, really. You could do both. Have the x86 directory that Linus suggests for shared files, then have the build system generate the symlinks for you. Could have arch/x86_64/kernel/common arch/x86_64/mm/common etc. symlinks. that point to arch/x86/kernel, arch/x86/mm etc. This way you know exactly which files are shared and which are not, which is basically impossible without a grep currently. You also get to do a single grep of all arch code. Best of both worlds? Or do I miss something? -- SUSE Labs, Novell Inc. Send instant messages to your online friends http://au.messenger.yahoo.com - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Thu, 2007-03-15 at 17:06 +0100, Andi Kleen wrote: > Well I just see a lot of pain from these patches but I doubt > they will avoid any bugs. If people don't compile test both > archs they will always likely break on another. There are lots > of subtle dependencies that are not expressed in the pathname > even after this intrusive operation (e.g. in the includes). > > That's just how it is. Or that's just how you see it. Yes, if you don't compile and test on both it can always break on the other arch. But this at least lets a developer know that what they are modifying requires a compile and test on both. The current way is more of a surprise that it breaks another arch. You're changing a file in arch/i386/... and you later get yelled at for breaking x86_64 ?? That's not negligence, that's blame for being ignorant of the interlocking dependencies. Where-as if you change the file in arch/x86 or include/asm-x86, you had better test both architectures. With the proposed patch set, what can break i386 by modifying something in arch/x86_64, or what can break x86_64 by modifying something in arch/i386? (not counting the unfinished pci shared code). > > If the architecture merging was ever done it would be likely > by extending arch/x86_64 to support (modern) 32bit. But this > change doesn't bring us any step closer to that goal. I'm not so sure about that. It at least gets people aware of the issues. If you expect x86_64 to start encompassing 32bit, then at least have all the files under arch/x86_64 and point the i386 there. Create something like a x86_common directory under arch/x86_64 and have that contain all the code that i386 can also share. Something that makes it cleaner and not the total hacks that we have today. > > I think it's just aesthetics -- i'm all for aesthetics but only > if it gives better software and doesn't impact other people who > want to get something real done; neither of this is the case here. Look, I sympathize with you. I'm not about to throw these patches at you and then walk away. I'll take responsibility for them (I've been warned by others that this will happen). If there's issues and hardships that cause you pain, feel free to kick some of the work over to me, and I'll help you out. I'll even accept it when you write to me saying "here! this is broken, and I think your arch/x86 crap is the cause. Fix it or else!". But I don't share your view that this doesn't give better software. And I think what I've done was real work. Hey, I'm fine with keeping this in arch/x86_64, in something called arch/x86_64/x86_common or something. As long as there's a distinct separation of what is shared and obvious that more testing needs to be done if modified. -- Steve - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Thu, 15 Mar 2007, Andi Kleen wrote: > > That's *exactly* what the patches do (except it's called "arch/x86", which > > is clearly the best option - please don't use "ia" _anywhere_ except for > > "ia64", since that's the only architecture that is really "intel > > architecture"). > > And i860 @) Yeah, well (and the i960).. I don't think it's very likely that we'll ever support that in the kernel ;) (There are others. i432 is the really *classic* intel architecture. It was such a spectacular failure that it's still remembered today, even though it was never actually used). > Well I just see a lot of pain from these patches but I doubt > they will avoid any bugs. If people don't compile test both > archs they will always likely break on another. There are lots > of subtle dependencies that are not expressed in the pathname > even after this intrusive operation (e.g. in the includes). Well, we'veliterally had bugs because people who modified i386 files didn't even *realize* that they were also used on x86-64. I do agree that we'd need to look at do the same thing for include files, for the same reason. Linus - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
> That's *exactly* what the patches do (except it's called "arch/x86", which > is clearly the best option - please don't use "ia" _anywhere_ except for > "ia64", since that's the only architecture that is really "intel > architecture"). And i860 @) > > On the downside, it's more ../../.. type stuff. On the upside, > > they're more cleanly separated and it's apparent what's going on. > > Seems nicer to me than drivers/ and kernel/ for stuff that's > > really arch specific, but shared between two arches. > > Absolutely. I'm agreeing violently. I just suspect not a lot of people > really looked at the patches.. Well I just see a lot of pain from these patches but I doubt they will avoid any bugs. If people don't compile test both archs they will always likely break on another. There are lots of subtle dependencies that are not expressed in the pathname even after this intrusive operation (e.g. in the includes). That's just how it is. If the architecture merging was ever done it would be likely by extending arch/x86_64 to support (modern) 32bit. But this change doesn't bring us any step closer to that goal. I think it's just aesthetics -- i'm all for aesthetics but only if it gives better software and doesn't impact other people who want to get something real done; neither of this is the case here. -Andi - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Thu, 15 Mar 2007, Martin Bligh wrote: > > Can't we move the shared files into a new shared arch/ subdirectory > (ia32_64 or whatever), and have them included from both places? That's *exactly* what the patches do (except it's called "arch/x86", which is clearly the best option - please don't use "ia" _anywhere_ except for "ia64", since that's the only architecture that is really "intel architecture"). > On the downside, it's more ../../.. type stuff. On the upside, > they're more cleanly separated and it's apparent what's going on. > Seems nicer to me than drivers/ and kernel/ for stuff that's > really arch specific, but shared between two arches. Absolutely. I'm agreeing violently. I just suspect not a lot of people really looked at the patches.. Linus - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
Linus Torvalds wrote: On Wed, 14 Mar 2007, Ingo Molnar wrote: and that's how i think unification of architectures should be done: move code into kernel/* and drivers/*, _not_ into another architecture. That way all architectures benefit. Don't be silly. Did you even *look* at the patches? We're talking about things like b/arch/x86/kernel/cpu/cpufreq/powernow-k8.h so please tell me what drugs you are on that say that we should move this into kernel/* and drivers/* and make all architectures benefit. It's *x86* specific, it's just that k8 exists both in 32-bit and 64-bit land. The file is *already* shared, it's just that right now it is not in a shared location. Right now it is in arch/i386/ (as if it was i386-specific) and then x86-64 includes it with: arch/x86_64/kernel/cpufreq/Makefile: # # Reuse the i386 cpufreq drivers # SRCDIR := ../../../i386/kernel/cpu/cpufreq and anybody who thinks this is "nice" simply has no taste at all. Can't we move the shared files into a new shared arch/ subdirectory (ia32_64 or whatever), and have them included from both places? At least then it's obvious they're shared, and it's a migration strategy to the shared arch that you want. On the downside, it's more ../../.. type stuff. On the upside, they're more cleanly separated and it's apparent what's going on. Seems nicer to me than drivers/ and kernel/ for stuff that's really arch specific, but shared between two arches. M. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
Linus Torvalds wrote: On Wed, 14 Mar 2007, Ingo Molnar wrote: and that's how i think unification of architectures should be done: move code into kernel/* and drivers/*, _not_ into another architecture. That way all architectures benefit. Don't be silly. Did you even *look* at the patches? We're talking about things like b/arch/x86/kernel/cpu/cpufreq/powernow-k8.h so please tell me what drugs you are on that say that we should move this into kernel/* and drivers/* and make all architectures benefit. It's *x86* specific, it's just that k8 exists both in 32-bit and 64-bit land. The file is *already* shared, it's just that right now it is not in a shared location. Right now it is in arch/i386/ (as if it was i386-specific) and then x86-64 includes it with: arch/x86_64/kernel/cpufreq/Makefile: # # Reuse the i386 cpufreq drivers # SRCDIR := ../../../i386/kernel/cpu/cpufreq and anybody who thinks this is nice simply has no taste at all. Can't we move the shared files into a new shared arch/ subdirectory (ia32_64 or whatever), and have them included from both places? At least then it's obvious they're shared, and it's a migration strategy to the shared arch that you want. On the downside, it's more ../../.. type stuff. On the upside, they're more cleanly separated and it's apparent what's going on. Seems nicer to me than drivers/ and kernel/ for stuff that's really arch specific, but shared between two arches. M. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Thu, 15 Mar 2007, Martin Bligh wrote: Can't we move the shared files into a new shared arch/ subdirectory (ia32_64 or whatever), and have them included from both places? That's *exactly* what the patches do (except it's called arch/x86, which is clearly the best option - please don't use ia _anywhere_ except for ia64, since that's the only architecture that is really intel architecture). On the downside, it's more ../../.. type stuff. On the upside, they're more cleanly separated and it's apparent what's going on. Seems nicer to me than drivers/ and kernel/ for stuff that's really arch specific, but shared between two arches. Absolutely. I'm agreeing violently. I just suspect not a lot of people really looked at the patches.. Linus - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
That's *exactly* what the patches do (except it's called arch/x86, which is clearly the best option - please don't use ia _anywhere_ except for ia64, since that's the only architecture that is really intel architecture). And i860 @) On the downside, it's more ../../.. type stuff. On the upside, they're more cleanly separated and it's apparent what's going on. Seems nicer to me than drivers/ and kernel/ for stuff that's really arch specific, but shared between two arches. Absolutely. I'm agreeing violently. I just suspect not a lot of people really looked at the patches.. Well I just see a lot of pain from these patches but I doubt they will avoid any bugs. If people don't compile test both archs they will always likely break on another. There are lots of subtle dependencies that are not expressed in the pathname even after this intrusive operation (e.g. in the includes). That's just how it is. If the architecture merging was ever done it would be likely by extending arch/x86_64 to support (modern) 32bit. But this change doesn't bring us any step closer to that goal. I think it's just aesthetics -- i'm all for aesthetics but only if it gives better software and doesn't impact other people who want to get something real done; neither of this is the case here. -Andi - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Thu, 15 Mar 2007, Andi Kleen wrote: That's *exactly* what the patches do (except it's called arch/x86, which is clearly the best option - please don't use ia _anywhere_ except for ia64, since that's the only architecture that is really intel architecture). And i860 @) Yeah, well (and the i960).. I don't think it's very likely that we'll ever support that in the kernel ;) (There are others. i432 is the really *classic* intel architecture. It was such a spectacular failure that it's still remembered today, even though it was never actually used). Well I just see a lot of pain from these patches but I doubt they will avoid any bugs. If people don't compile test both archs they will always likely break on another. There are lots of subtle dependencies that are not expressed in the pathname even after this intrusive operation (e.g. in the includes). Well, we'veliterally had bugs because people who modified i386 files didn't even *realize* that they were also used on x86-64. I do agree that we'd need to look at do the same thing for include files, for the same reason. Linus - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Thu, 2007-03-15 at 17:06 +0100, Andi Kleen wrote: Well I just see a lot of pain from these patches but I doubt they will avoid any bugs. If people don't compile test both archs they will always likely break on another. There are lots of subtle dependencies that are not expressed in the pathname even after this intrusive operation (e.g. in the includes). That's just how it is. Or that's just how you see it. Yes, if you don't compile and test on both it can always break on the other arch. But this at least lets a developer know that what they are modifying requires a compile and test on both. The current way is more of a surprise that it breaks another arch. You're changing a file in arch/i386/... and you later get yelled at for breaking x86_64 ?? That's not negligence, that's blame for being ignorant of the interlocking dependencies. Where-as if you change the file in arch/x86 or include/asm-x86, you had better test both architectures. With the proposed patch set, what can break i386 by modifying something in arch/x86_64, or what can break x86_64 by modifying something in arch/i386? (not counting the unfinished pci shared code). If the architecture merging was ever done it would be likely by extending arch/x86_64 to support (modern) 32bit. But this change doesn't bring us any step closer to that goal. I'm not so sure about that. It at least gets people aware of the issues. If you expect x86_64 to start encompassing 32bit, then at least have all the files under arch/x86_64 and point the i386 there. Create something like a x86_common directory under arch/x86_64 and have that contain all the code that i386 can also share. Something that makes it cleaner and not the total hacks that we have today. I think it's just aesthetics -- i'm all for aesthetics but only if it gives better software and doesn't impact other people who want to get something real done; neither of this is the case here. Look, I sympathize with you. I'm not about to throw these patches at you and then walk away. I'll take responsibility for them (I've been warned by others that this will happen). If there's issues and hardships that cause you pain, feel free to kick some of the work over to me, and I'll help you out. I'll even accept it when you write to me saying here! this is broken, and I think your arch/x86 crap is the cause. Fix it or else!. But I don't share your view that this doesn't give better software. And I think what I've done was real work. Hey, I'm fine with keeping this in arch/x86_64, in something called arch/x86_64/x86_common or something. As long as there's a distinct separation of what is shared and obvious that more testing needs to be done if modified. -- Steve - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
Ingo Molnar wrote: * Linus Torvalds [EMAIL PROTECTED] wrote: symbolic links perhaps? In that case i'd also introduce a common naming scheme: x86_early_printk.c - to make sure we know it right away that those files are bi-arch. Hey, I know! This is a radical idea, but what if we put the name at the head of the file, and called it arch/x86/kernel/early_printk.c instead? Then we could teach each of i386 and x86-64 to include it from that area, and we could put other shared files under the same directory hierarchy so that it would be easy to see which ones are shared? that is nice too, but it has some disadvantages as well in practice. For example i often want to see 'everything' that belongs to an arch in just one subdirectory. That way one can grep it for example, instead of having to grep two separate places. Symlinks would be fine for that, but an explicit split not really i think - unless we can get some really significant chunk of code into that hierarchy, so that it makes functional /sense/ to look at it in isolation. with the prefix suggestion we can keep these 'shared' files merged in a single, main functional tree (x86_64), but still have them marked in the VFS as being shared. But ... either way is fine to me - no strong feelings, really. You could do both. Have the x86 directory that Linus suggests for shared files, then have the build system generate the symlinks for you. Could have arch/x86_64/kernel/common arch/x86_64/mm/common etc. symlinks. that point to arch/x86/kernel, arch/x86/mm etc. This way you know exactly which files are shared and which are not, which is basically impossible without a grep currently. You also get to do a single grep of all arch code. Best of both worlds? Or do I miss something? -- SUSE Labs, Novell Inc. Send instant messages to your online friends http://au.messenger.yahoo.com - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Thu, 2007-03-15 at 12:47 -0400, Steven Rostedt wrote: On Thu, 2007-03-15 at 17:06 +0100, Andi Kleen wrote: Well I just see a lot of pain from these patches but I doubt they will avoid any bugs. If people don't compile test both archs they will always likely break on another. There are lots of subtle dependencies that are not expressed in the pathname even after this intrusive operation (e.g. in the includes). [...] With the proposed patch set, what can break i386 by modifying something in arch/x86_64, or what can break x86_64 by modifying something in arch/i386? (not counting the unfinished pci shared code). Oops, sorry, you did say in the includes. Yeah, that holds the same crap that I'm talking about. $ cat include/asm-i386/stacktrace.h #include asm-x86_64/stacktrace.h Which isn't something that I would like to stay either. So it's just something else that needs to be fixed. Not something that will circumvent what the current patch set is trying to do. -- Steve - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
Oops, sorry, you did say in the includes. Yeah, that holds the same crap that I'm talking about. It's a simple and obvious solution that does exactly what it is supposed to do. Why do you call it crap? -Andi - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Thu, 2007-03-15 at 18:01 +0100, Andi Kleen wrote: Oops, sorry, you did say in the includes. Yeah, that holds the same crap that I'm talking about. It's a simple and obvious solution that does exactly what it is supposed to do. Why do you call it crap? Yes, it's a simple solution. The reason I called it crap was because it does what the other hacks do. It hides from the developer a dependency that can create bugs in the other arch, and is not IMNSHO clean. When one works on one arch, you should not need to be worried about breaking other archs when you stay within the arch/ and asm-arch directories. -- Steve - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
You could do both. Have the x86 directory that Linus suggests for shared files, then have the build system generate the symlinks for you. Symlinks are usually a bad idea because they tend to not work with objdirs. We did that in 2.4. -Andi - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Mar 15 2007 08:59, Linus Torvalds wrote: On Thu, 15 Mar 2007, Martin Bligh wrote: Can't we move the shared files into a new shared arch/ subdirectory (ia32_64 or whatever), and have them included from both places? That's *exactly* what the patches do (except it's called arch/x86, which is clearly the best option - please don't use ia _anywhere_ except for ia64, since that's the only architecture that is really intel architecture). IA32 = Intel/AMD 32 :) Jan -- - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Wed, 2007-03-14 at 01:08 -0400, Steven Rostedt wrote: [Hopefully fixed email client to make it to the list this time] [This series has changed by using git-diff -M] snip Seems appropriate, but I really don't care what it's called. One thing about this name, is that typing arch/x86Tab doesn't tab complete x86_64 anymore. But if you can think of something better, I'd be happy to apply it. sorry for being so late, but about what it could be called, well, what about common_x86 or common/x86 or something? snip -- Steve PS. Sorry for the spam. I need to figure out how to tame quilt mail! -- - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Thu, 15 Mar 2007, Steven Rostedt wrote: On Thu, 2007-03-15 at 17:06 +0100, Andi Kleen wrote: Well I just see a lot of pain from these patches but I doubt they will avoid any bugs. If people don't compile test both archs they will always likely break on another. There are lots of subtle dependencies that are not expressed in the pathname even after this intrusive operation (e.g. in the includes). That's just how it is. Or that's just how you see it. In the future it is likely that x86_64 will significantly deviate from i386. i386 is going to be gradually abandoned because it does not support the ever larger memory sizes and be mainly used for embedded devices. x86_64 is going to acquire more functionality that will not be available for i386. We plan f.e. to add virtual memmap support for x86_64. Virtual memmap support may require a large chunk of virtual memory space that is not available on i386. Its not good to have to deal with i386 issues when doing x86_64 arch development. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Mar 14 2007 21:21, Andi Kleen wrote:
>
>> also, having the x32_ and x64_ prefix is a painful daily reminder for
>> all of us changing the architecture that 'this stuff needs to be
>> unified!'.
>
>We would probably stuck with that forever and it just looks ugly.
>Non paravirt xen uses such a setup and I always hated it.
>
>Besides the architecture is called x86_64
If you don't like x{32,64}_ suffixes, _{32,64} suffixes would also be a
possibility. powerpc/mm/init_32.c for example does it that way.
Jan
--
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
> the basic dynamics of legacies does not change if we have only 50% of > them: right now x86_64 is just growing its own set of legacies, at the > same rate as i386 did it 10 years ago. Modern system are much more similar to each other than older systems due to Windows forcing them and they are using much more hardware. For example we'll never have the tons of weird APIC errata again old platforms had. And the 64bit APIC code is already much cleaner because of this and this made new work (like Eric's rework which only happened partially on i386) much easier. Also 64bit doesn't support any weirdnesses like visws,voyager which probably should have been ripped out long ago because they cause large issues in the code and are practically unused (I bet both have userbases I can count on my fingers) > practice: those legacies will quickly necessiate the _same_ kinds of > abstractions that allow the flexible injection of hardware-dependent > quirks. In another 5 years the x86_64 tree will end up looking and That's not my experience. > behaving _just like the i386_ tree, the only difference will be less > compatibility. (In fact, it will likely look worse because currently our > efforts are 50% split between i386 and x86_64, and the random That's also not how it works. Typically bug fixes are applied to both. > Also, 90% of our users are still running 32bit kernels _even on 64-bit > capable hardware_, so we might as well prepare ourselves for a really > long march towards a pure 64-bit world. (Which will likely never come.) 32bit is not the problem, the problem is old platforms/weird not quite PC platforms (which don't exist in the 64bit world)/very old CPUs. 32bit has some ugly legacies (like vm86) but they are handleable. The old bugs aren't imho. When a new unified port was done it could be done for modern x86 only, with arch/i386 kept for the old legacy stuff. That would be fine for me. The trouble is just that I'm not sure that distributions would actually ship it e.g. because not supporting P5s anymore might be still too radical. And if it was not shipped then it would be useless. Besides it's would be a lot of work and frankly not very interesting work. -Andi - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
* Andi Kleen <[EMAIL PROTECTED]> wrote: > > Andrew's laptop only half a dozen times! ;) But .. in the long run, > > it's alot easier to think about unified code. 32-bit x86 will > > certainly stay with us for at least 10-20 years, and the best model > > for maintainance is having one codebase. > > Not sure -- i'm often glad I don't have to care about all the old > 32bit systems on x86-64. [...] the basic dynamics of legacies does not change if we have only 50% of them: right now x86_64 is just growing its own set of legacies, at the same rate as i386 did it 10 years ago. That makes little difference in practice: those legacies will quickly necessiate the _same_ kinds of abstractions that allow the flexible injection of hardware-dependent quirks. In another 5 years the x86_64 tree will end up looking and behaving _just like the i386_ tree, the only difference will be less compatibility. (In fact, it will likely look worse because currently our efforts are 50% split between i386 and x86_64, and the random differences between the two arches are wasting developer resources.) so we might as well unify the two trees and /learn/ from i386's legacies, while integrating them. Those legacies, by the rule of large numbers, will revisit x86_64 too (or have already visited it). We already have per-APIC-version quirks, per CPU model quirks, etc., etc. The main cost of a quirk is the abstraction it necessiates, not the quirk handler itself. (which, once the framework is there, is modular) Also, 90% of our users are still running 32bit kernels _even on 64-bit capable hardware_, so we might as well prepare ourselves for a really long march towards a pure 64-bit world. (Which will likely never come.) Ingo - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
> also, having the x32_ and x64_ prefix is a painful daily reminder for > all of us changing the architecture that 'this stuff needs to be > unified!'. We would probably stuck with that forever and it just looks ugly. Non paravirt xen uses such a setup and I always hated it. Besides the architecture is called x86_64 I don't see a problem with step by step unifying what makes sense in the current setup. -Andi - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
* Ingo Molnar <[EMAIL PROTECTED]> wrote: > > Well, I'd like it to be 100% _eventually_, and just unify the > > architectures. > > ok, having a single bi-arch final tree is indeed intriquing and i didnt > realize that you were suggesting that. [...] > > [...] But this really scares the sh*t out of me. [...] i think the sanest (and pretty much only) way to do it would be to unify it _now_, in a single atomic step, safely and in a brute-force way: create arch/x86/ 'unify' arch/i386/ and arch/x86_64/ into arch/x86/ by doing a brute-force prefix solution: arch/i386/kernel/process.c => arch/x86/kernel/x32_process.c arch/x86_64/kernel/process.c => arch/x86/kernel/x64_process.c create bi-arch Makefiles and enable the building of each arch. obviously, for files that are already totally identical across the two arches, the x32_foo.c and x64_foo.c can be changed to foo.c, right away. nothing else would be done in this step. We'd remove arch/i386/ and arch/x86_64/ after that. new files for new functionality can only be added if they are bi-arch. from this first 'brutal', 'flag day' intrusive step on, every further step would be more finegrained and per-file: an x32_bar.c and x64_bar.c file gets merged into bar.c. About 80-90% of our .c files have the same fundamental purpose in both arches, so this setup makes the most sense to me: first move them 'close' to each other and force them to be part of the same basic build architecture, then 'unify' them. this way we'd also obviously see the costs of not having unified files: we'd have to change both the x32_ and the x64_ file for the same thing - while when they are unified, only one change is needed. It's easy to forget that when the arch directories are separate. also, unifying two files will also be quite gratifying: one of the files vanishes, making up for a sail-through-lkml review. also, this way it's harder for the two trees (and the shared tree) to get out of sync. If we did 3 separate hierarchies we'd have mixed state pretty quickly. also, having the x32_ and x64_ prefix is a painful daily reminder for all of us changing the architecture that 'this stuff needs to be unified!'. hm? Ingo - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
> the x86_64 and i386 trees have diverged quite a bit though, so this will > be a major logistical undertaking. And with Andi opposed to > fundamentally it it also lacks a bit of manpower i guess :-/ I'm not fundamentally opposed, just sceptical on the effort:gain ratio. > Andrew's laptop only half a dozen times! ;) But .. in the long run, it's > alot easier to think about unified code. 32-bit x86 will certainly stay > with us for at least 10-20 years, and the best model for maintainance is > having one codebase. Not sure -- i'm often glad I don't have to care about all the old 32bit systems on x86-64. I have no problem with modern 32bit x86 systems, but old x86 tends to be very different from new x86. And all the old erratas are scary. -Andi - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
* Linus Torvalds <[EMAIL PROTECTED]> wrote: > > of code. i386 is 87847 lines of code, x86_64 is 40978 lines of code, > > a total of 128825. That means we move about 10% of the code. Not > > insignificant but not earth-shattering either. With alot more effort > > (and testing) we could realistically go up to maybe 20% - but that's > > still a bit low to spread out all the files, isnt it? > > Well, I'd like it to be 100% _eventually_, and just unify the > architectures. ok, having a single bi-arch final tree is indeed intriquing and i didnt realize that you were suggesting that. (I had the impression that arch/x86/ was more of a 'common library' thing, not a target architecture. What felt weird to me was having 3 separate hierarchies - but as long as it's just a temporary state it's OK.) > We've now done that both for S/390 and POWER, and I think in both > cases it's been a clear win. So it's not like this is even a radical > idea. the x86_64 and i386 trees have diverged quite a bit though, so this will be a major logistical undertaking. And with Andi opposed to fundamentally it it also lacks a bit of manpower i guess :-/ > There really is almost nothing in i386 that shouldn't be supported on > x86-64 too, unless it literally is the actual low-level asm files and > vm86 mode support (which in turn is best left as just a config option > that would just *depend* on 32-bit, so even that could sanely be > represented in a shared tree without any real downside at all). yeah. But this really scares the sh*t out of me. I already tried to unify some of the most fragile lowlevel bits recently: for example the SMP bootup, TSC sync and APIC initialization sequences were totally different on x86 and x86_64. And those kind of random deviations have spread all around the tree. But ... my experience has been pretty positive: touching both codebases at once tends to dust off old code and tends to fix more bugs. And in the process of doing that we broke Andrew's laptop only half a dozen times! ;) But .. in the long run, it's alot easier to think about unified code. 32-bit x86 will certainly stay with us for at least 10-20 years, and the best model for maintainance is having one codebase. Another practical complication is that even for modular stuff, sometimes x86_64 has the better code, sometimes i386. But ... the more i think about it the more i like it. -m32 certainly works fine and does the right thing. Ingo - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
Andi Kleen wrote: > Only do it where it makes sense and is not too intrusive. > Redoing the whole port just for lguest64 is probably not a good idea. Well, at some point Xen is going to be 64-bit. We need a 64-bit paravirt_ops, and it looks to me that 90% of the entrypoints will be more or less identical for 32 and 64 bit x86, so it would be unfortunate to not share it. The existing technique for sharing stuff between 32 and 64 bits is pretty damn ugly, and I think Steven is on to something with this patch. Though I'm definitely sympathetic to the concern that it will break a pile of outstanding patches. J - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Wed, Mar 14, 2007 at 09:36:35AM -0400, Steven Rostedt wrote: > On Wed, 2007-03-14 at 14:05 +0100, Andi Kleen wrote: > > > The thing is others and I (and you) are working on getting paravirt_ops > > > working for x86_64. There's a lot of overlap between i386 and x86_64. > > > Right now the i386 is ahead of x86_64 and the code seems to be put more > > > in the arch/i386 arch. So now we are going to introduce a > > > new ../../i386 hack to get to a shared paravirt_shared.c(?). Or do we > > > > What would you like exactly to share? > > I'm still working on that. Others have pressured us into consolidating > as much usable code for i386 into x86_64. This may turn out not to be > too feasible anyway. Only do it where it makes sense and is not too intrusive. Redoing the whole port just for lguest64 is probably not a good idea. -Andi - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Wed, Mar 14, 2007 at 11:36:08AM +0100, Andi Kleen wrote: > Steven Rostedt <[EMAIL PROTECTED]> writes: > > > > So I spent last night hacking up something to try to make a common ground > > for all code that is shared between x86_64 and i386. I called this > > > >arch/x86 > > NACK. I think the current ways work just fine. In my experience, the current situation is a pain in the ass. There are at least 3 different ways how arch/x86_64/ uses arch/i386/ code [1], and at least 2 different ways how arch/i386/ uses arch/x86_64/ So whenever you touch one file in arch/i386/ or arch/x86_64/, there are 5 different ways how this code could be used from the other architecture. I'd even suggest the following: - move all files under arch/x86_64/ that are also used under arch/i386/ to arch/i386/ and then - mv arch/x86_64 arch/i386/mach-x86_64 The associated Makefile and Kconfig changes shouldn't be a problem, and after this it will become much easier to see where code gets used. > -Andi cu Adrian [1] Steven forgot in his list the fifth one: obj-$(CONFIG_MTRR) += ../../i386/kernel/cpu/mtrr/ The problem with this variant is that different from the other 4 ways, grep'ing for a filename doesn't find it... -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Wed, 14 Mar 2007, Ingo Molnar wrote: > > then i decided to analyze the patches: currently they move 13452 lines > of code. i386 is 87847 lines of code, x86_64 is 40978 lines of code, a > total of 128825. That means we move about 10% of the code. Not > insignificant but not earth-shattering either. With alot more effort > (and testing) we could realistically go up to maybe 20% - but that's > still a bit low to spread out all the files, isnt it? Well, I'd like it to be 100% _eventually_, and just unify the architectures. We've now done that both for S/390 and POWER, and I think in both cases it's been a clear win. So it's not like this is even a radical idea. The POWER architecture merge was actually done exactly the incremental way, one file or directory at a time, and seemed to work out fine. So while I'd like 100%, I'd be happy to even just get started with the really obvious stuff. And the stuff that we *already* share certainly fall under that "really obvious" label. > So i thought it's a better idea to continue with the current more > finegrained scheme of sharing some files between the architectures by > having arch/x86_64 be the 'main' repository, with i386 inheriting them > back, instead of spreading out the files? That's really illogical, because historially, i386 was the main one, so you'd have to either: - have a really strange mix OR - move files around just to share them. So I'd much rather have just a totally new architecture, and hope that we can migrate more and more to it. Whether we ever get to the POWER situation where only some really odd-ball special cases are still using ppc or not, who knows? It might be, for example, that only the odder i386 cases (ie the "non-PC" subarchitectures - Voyager, Summit, NUMAQ and the like) would stay in i386. There really is almost nothing in i386 that shouldn't be supported on x86-64 too, unless it literally is the actual low-level asm files and vm86 mode support (which in turn is best left as just a config option that would just *depend* on 32-bit, so even that could sanely be represented in a shared tree without any real downside at all). Linus - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
* Linus Torvalds <[EMAIL PROTECTED]> wrote: > > symbolic links perhaps? In that case i'd also introduce a common > > naming scheme: x86_early_printk.c - to make sure we know it right > > away that those files are bi-arch. > > Hey, I know! This is a radical idea, but what if we put the name at > the head of the file, and called it > > arch/x86/kernel/early_printk.c > > instead? Then we could teach each of i386 and x86-64 to include it > from that area, and we could put other shared files under the same > directory hierarchy so that it would be easy to see which ones are > shared? that is nice too, but it has some disadvantages as well in practice. For example i often want to see 'everything' that belongs to an arch in just one subdirectory. That way one can grep it for example, instead of having to grep two separate places. Symlinks would be fine for that, but an explicit split not really i think - unless we can get some really significant chunk of code into that hierarchy, so that it makes functional /sense/ to look at it in isolation. with the prefix suggestion we can keep these 'shared' files merged in a single, main functional tree (x86_64), but still have them marked in the VFS as being shared. But ... either way is fine to me - no strong feelings, really. Ingo - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
* Linus Torvalds <[EMAIL PROTECTED]> wrote: > Did you even *look* at the patches? yes. I am strongly in favor of sharing code - i recently introduced arch/x86_64/kernel/tsc_sync.c that is shared by i386 too. So first i wrote a draft email where i told Andi that he's on crack to NACK it so brutally and that we should clearly do arch/x86 ;-) then i decided to analyze the patches: currently they move 13452 lines of code. i386 is 87847 lines of code, x86_64 is 40978 lines of code, a total of 128825. That means we move about 10% of the code. Not insignificant but not earth-shattering either. With alot more effort (and testing) we could realistically go up to maybe 20% - but that's still a bit low to spread out all the files, isnt it? in comparison, arch/powerpc is 159099 lines of code, arch/ppc is 106598 lines of code. So the sharing factor is quite a bit higher there. I dont think we could ever reach that with x86. So i thought it's a better idea to continue with the current more finegrained scheme of sharing some files between the architectures by having arch/x86_64 be the 'main' repository, with i386 inheriting them back, instead of spreading out the files? but, whichever variant happens, it's certainly fine to me :) Ingo - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Wed, 14 Mar 2007, Steven Rostedt wrote: > > That's created at build time. But I don't see anywhere in a freshly > cloned repo or fresh untar of the linux tarball, where there exists any > symbolic links. There are none. Symlinks embedded in the source tree tend to be hard to maintain: you can traditionally not send patches to add/change/move/remove them, and not everybody can even import them at all (ie some people have been so damaged by CVS that they maintain their kernels in it - I'm trying my best to be a humanitarian and rid the world of the scourge that is CVS, but I'm not sure I can undo the untold mental damage wrought by it over decades of quiet suffering). With git, you can track symlinks and send them as patches, but we've not really had a huge reason to do so. They are easy enough to generate from Makefiles if required, and quite often you don't really need to anyway (ie the "symlink" is often just a make rule, like the SRCDIR := ../../../i386/kernel/cpu/cpufreq line I quoted earlier). So I'd rather not even start using symlinks unless there is some really good reason. We can continue to just use Makefiles. Linus - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Wed, 2007-03-14 at 17:33 +0100, Jan Engelhardt wrote: > On Mar 14 2007 10:46, Steven Rostedt wrote: > > > >> symbolic links perhaps? In that case i'd also introduce a common naming > >> scheme: x86_early_printk.c - to make sure we know it right away that > >> those files are bi-arch. > > > >Does the Linux code tree already support sym links? IOW, are there > >already sym links in the code tree? (/me probably should just look ;) > > /usr/src/linux/include/asm -> /usr/src/linux/include/asm-i386 That's created at build time. But I don't see anywhere in a freshly cloned repo or fresh untar of the linux tarball, where there exists any symbolic links. -- Steve - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Wed, 14 Mar 2007, Jan Engelhardt wrote: > > > >Seems appropriate, but I really don't care what it's called. One thing about > >this name, is that typing arch/x86 doesn't tab complete x86_64 anymore. > >But if you can think of something better, I'd be happy to apply it. > > 80x86 > 8086 > ia32 <- I'd go with this for lack of more entries on this list. But the whole point is that this is *not* a 32-bit file. It's a file *shared* between 32-bit and 64-bit kernels. Which is why plain "x86" is the right name. Yeah, Intel calls it "IA" these days, but that's just crazy speak (and probably a combination of a reaction against "amd64" and a way to hide the embarrassment that is "ia64" by confusing the naming so much that one day people won't remember that there was _another_ IA that was a total failure). Linus - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Wed, 14 Mar 2007, Ingo Molnar wrote: > > symbolic links perhaps? In that case i'd also introduce a common naming > scheme: x86_early_printk.c - to make sure we know it right away that > those files are bi-arch. Hey, I know! This is a radical idea, but what if we put the name at the head of the file, and called it arch/x86/kernel/early_printk.c instead? Then we could teach each of i386 and x86-64 to include it from that area, and we could put other shared files under the same directory hierarchy so that it would be easy to see which ones are shared? Crazy idea, I know. But I think I saw somebody send out a patch kind of like that, so maybe this common naming scheme could be as simple as just applying a few patches once they got cleaned up and re-organized some more. Linus - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Mar 14 2007 10:46, Steven Rostedt wrote: > >> symbolic links perhaps? In that case i'd also introduce a common naming >> scheme: x86_early_printk.c - to make sure we know it right away that >> those files are bi-arch. > >Does the Linux code tree already support sym links? IOW, are there >already sym links in the code tree? (/me probably should just look ;) /usr/src/linux/include/asm -> /usr/src/linux/include/asm-i386 Jan -- - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Wed, 14 Mar 2007, Ingo Molnar wrote: > > and that's how i think unification of architectures should be done: move > code into kernel/* and drivers/*, _not_ into another architecture. That > way all architectures benefit. Don't be silly. Did you even *look* at the patches? We're talking about things like b/arch/x86/kernel/cpu/cpufreq/powernow-k8.h so please tell me what drugs you are on that say that we should move this into kernel/* and drivers/* and make all architectures benefit. It's *x86* specific, it's just that k8 exists both in 32-bit and 64-bit land. The file is *already* shared, it's just that right now it is not in a shared location. Right now it is in arch/i386/ (as if it was i386-specific) and then x86-64 includes it with: arch/x86_64/kernel/cpufreq/Makefile: # # Reuse the i386 cpufreq drivers # SRCDIR := ../../../i386/kernel/cpu/cpufreq and anybody who thinks this is "nice" simply has no taste at all. Linus - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Wed, 14 Mar 2007, Andi Kleen wrote: > > NACK. I think the current ways work just fine. Andi, the current ways do *not* work just fine. I don't understand why you have problems with obvious cleanups. You also nack'ed the file movement to at least make this kind of thing consistent (ie the header file movement in commit 2272b0e03) to at least only have symlinks pointing one way. The current situation is horrible. We've had bugs simply because people didn't even *realize* that a file in arch/i386/ was also used by x86-64. Linus - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Wed, 2007-03-14 at 14:41 +0100, Ingo Molnar wrote: > hm. Do you have any numbers handy - what is the end-result of your > unification work, how many lines of code were unified, compared to the > total body of code in i386 and x86_64? Well, I wasn't combining code that wasn't already combined :) Just moving code that was referenced by another arch to a common directory that showed the code is shared. So, I moved a total of 44 files that were shared. Some of these were in places where a full directory was shared (the mtrr code). And this doesn't count the Makefiles and Kconfig files that were also modified. There were only three files with the #include "../..//" code. These three where early_printk.c, tsc_sync.c and msr-on-cpu.c. The rest is referenced by the Makefiles. This still doesn't make it easy to find functions via TAGS or search scripts. If you looked at the 18/18 patch, it has a list of the moved files, with the exception of the speedstep-lib.h, which was moved in it's own file, and those that referenced that header. > symbolic links perhaps? In that case i'd also introduce a common naming > scheme: x86_early_printk.c - to make sure we know it right away that > those files are bi-arch. Does the Linux code tree already support sym links? IOW, are there already sym links in the code tree? (/me probably should just look ;) So should we have an effort to label the shared code that's already shared. As Andi stated, he doesn't like "large scale" renaming since that doesn't "fix a single bug", and will only "just cause pain". Although I disagree if in the long run it will make it easier to work with. Once one knows about the crazy linking going on then it's not much of a problem, but what about all those that will have to go through this learning curve. The problem I have with the current approach is that it just isn't clean. Yes it "works", but it still is a hack. And if I do need to write code that will be shared among the two archs (still don't know for sure if this will be the case), I would like to have a clean method in doing it. I don't care what the final solution is, as long as it is clean and not a hack. -- Steve - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
* Steven Rostedt <[EMAIL PROTECTED]> wrote: > > i agree. We've recently factored out quite a bit of common code > > between i386 and x86_64 recently: genirq, gtod/clocksource and > > clockevents. > > But those are things that can mostly be shared across all archs. yeah. > > and that's how i think unification of architectures should be done: > > move code into kernel/* and drivers/*, _not_ into another > > architecture. That way all architectures benefit. > > I'm OK with it, although I did waste a lot of effort making those > patches (But I speak better in C than in English, or any other verbal > language, so it wasn't that bad). But if you look at the code that was > merged, I'm not sure many other archs will benefit. How many archs use > mtrr's? Perhaps these still can go into the drivers directory with a > bit of work. Don't know, I'm not that close to that code to be sure, > and don't have the time to find out ATM. hm. Do you have any numbers handy - what is the end-result of your unification work, how many lines of code were unified, compared to the total body of code in i386 and x86_64? > But at least there needs to be a more common way to share files > between the two archs. Having a file with just a single line of: > > #include "../../i386/kernel/mycommoncode.c" > > is not that elegant. The make files are, perhaps, a bit better. yeah, that #include file thing for early_printk.c is just gross. > Another thing that happens a lot with these shared funcs in these > files, is finding them . Since "make TAGS" doesn't bother to check > i386 when run with ARCH=x86_64. The first time I searched for > early_printk while developing i386 took me an hour, since my search > scripts don't check other archs (I've changed that since). I thought > that the function was one of these macro created functions, and was > non-arch specific (didn't look into arch). > > So, when creating new shared code, what's the "proper" way? symbolic links perhaps? In that case i'd also introduce a common naming scheme: x86_early_printk.c - to make sure we know it right away that those files are bi-arch. Ingo - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Wed, 2007-03-14 at 14:05 +0100, Andi Kleen wrote: > > The thing is others and I (and you) are working on getting paravirt_ops > > working for x86_64. There's a lot of overlap between i386 and x86_64. > > Right now the i386 is ahead of x86_64 and the code seems to be put more > > in the arch/i386 arch. So now we are going to introduce a > > new ../../i386 hack to get to a shared paravirt_shared.c(?). Or do we > > What would you like exactly to share? I'm still working on that. Others have pressured us into consolidating as much usable code for i386 into x86_64. This may turn out not to be too feasible anyway. > > > just continue on keeping the x86_64 as a separate entity, with a lot of > > duplicate code? > > When it makes sense to share the code it can be shared. That is already > done today. But the existing Makefile mechanisms work fine for that. > > Making sense: > - There is actually a lot of shared code > - The supported hardware is the same (remember x86-64 is only for > modern x86 hardware, while i386 has a much longer legacy) > - No ifdefs Thanks, I'll keep this in mind. -- Steve - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Wed, 2007-03-14 at 13:53 +0100, Ingo Molnar wrote: > * Andi Kleen <[EMAIL PROTECTED]> wrote: > > > Steven Rostedt <[EMAIL PROTECTED]> writes: > > > > > > So I spent last night hacking up something to try to make a common > > > ground for all code that is shared between x86_64 and i386. I > > > called this > > > > > >arch/x86 > > > > NACK. I think the current ways work just fine. > > i agree. We've recently factored out quite a bit of common code between > i386 and x86_64 recently: genirq, gtod/clocksource and clockevents. But those are things that can mostly be shared across all archs. > > and that's how i think unification of architectures should be done: move > code into kernel/* and drivers/*, _not_ into another architecture. That > way all architectures benefit. > I'm OK with it, although I did waste a lot of effort making those patches (But I speak better in C than in English, or any other verbal language, so it wasn't that bad). But if you look at the code that was merged, I'm not sure many other archs will benefit. How many archs use mtrr's? Perhaps these still can go into the drivers directory with a bit of work. Don't know, I'm not that close to that code to be sure, and don't have the time to find out ATM. But at least there needs to be a more common way to share files between the two archs. Having a file with just a single line of: #include "../../i386/kernel/mycommoncode.c" is not that elegant. The make files are, perhaps, a bit better. Another thing that happens a lot with these shared funcs in these files, is finding them . Since "make TAGS" doesn't bother to check i386 when run with ARCH=x86_64. The first time I searched for early_printk while developing i386 took me an hour, since my search scripts don't check other archs (I've changed that since). I thought that the function was one of these macro created functions, and was non-arch specific (didn't look into arch). So, when creating new shared code, what's the "proper" way? -- Steve - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
> The thing is others and I (and you) are working on getting paravirt_ops > working for x86_64. There's a lot of overlap between i386 and x86_64. > Right now the i386 is ahead of x86_64 and the code seems to be put more > in the arch/i386 arch. So now we are going to introduce a > new ../../i386 hack to get to a shared paravirt_shared.c(?). Or do we What would you like exactly to share? > just continue on keeping the x86_64 as a separate entity, with a lot of > duplicate code? When it makes sense to share the code it can be shared. That is already done today. But the existing Makefile mechanisms work fine for that. Making sense: - There is actually a lot of shared code - The supported hardware is the same (remember x86-64 is only for modern x86 hardware, while i386 has a much longer legacy) - No ifdefs -Andi - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
* Andi Kleen <[EMAIL PROTECTED]> wrote: > Steven Rostedt <[EMAIL PROTECTED]> writes: > > > > So I spent last night hacking up something to try to make a common > > ground for all code that is shared between x86_64 and i386. I > > called this > > > >arch/x86 > > NACK. I think the current ways work just fine. i agree. We've recently factored out quite a bit of common code between i386 and x86_64 recently: genirq, gtod/clocksource and clockevents. and that's how i think unification of architectures should be done: move code into kernel/* and drivers/*, _not_ into another architecture. That way all architectures benefit. Ingo - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Wed, 2007-03-14 at 11:36 +0100, Andi Kleen wrote: > Steven Rostedt <[EMAIL PROTECTED]> writes: > > > > So I spent last night hacking up something to try to make a common ground > > for all code that is shared between x86_64 and i386. I called this > > > >arch/x86 > > NACK. I think the current ways work just fine. Fine. The thing is others and I (and you) are working on getting paravirt_ops working for x86_64. There's a lot of overlap between i386 and x86_64. Right now the i386 is ahead of x86_64 and the code seems to be put more in the arch/i386 arch. So now we are going to introduce a new ../../i386 hack to get to a shared paravirt_shared.c(?). Or do we just continue on keeping the x86_64 as a separate entity, with a lot of duplicate code? I'm OK with either way. I just wanted to set a standard while it's still early enough to set one. -- Steve - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
Hi, I am newbie developing a routing application which needs three features; 1. if the fib lookup fails, my application needs to know about using preferably netlink, -- any direction to some sample code or files in the kernel??? 2. I need a counter recording the hits a fib entry is chosen for packet forwarding. Is it the function fn_trie_lookup in file /net/ipv4/fib_trie.c i need to modify and look into? -- ?? and therefore is it safe to add a int counter in fib_node w/o breaking the build?? 2. need to read the entire fib, i.e dump the fib to the user space, -- again any direction to some sample code etc.. i have seen a fn in /source/net/ipv4/fib_frontend.c , which looks like is supposed to dump the routes but not sure how to use it.. line no.581 int inet_dump_fib(struct sk_buff *skb, struct netlink_callback *cb) Also; fn_trie_dump() in /net/ipv4/fib_trie.c This could be an inappropriate list for the above questions, if it is request for some direction on the appropriate mailing list Appreciate any help. Thanking you, Sujay - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
Steven Rostedt <[EMAIL PROTECTED]> writes: > > So I spent last night hacking up something to try to make a common ground > for all code that is shared between x86_64 and i386. I called this > >arch/x86 NACK. I think the current ways work just fine. -Andi - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Mar 14 2007 01:08, Steven Rostedt wrote: > >So I spent last night hacking up something to try to make a common ground >for all code that is shared between x86_64 and i386. I called this > > arch/x86 > >Seems appropriate, but I really don't care what it's called. One thing about >this name, is that typing arch/x86 doesn't tab complete x86_64 anymore. >But if you can think of something better, I'd be happy to apply it. 80x86 8086 ia32 <- I'd go with this for lack of more entries on this list. i486 i86pc <- Solely listed for completes. No Solarisisms in Linux please :p ix86 x32<- Blame Microsoft for introducing "x64" x86_32 <- Will interfere with your tabbing. >So the moves are simply renames, with the slight exception >of files that hold the speedstep-lib.h file. This file was moved from the >arch/i386/kerne/cpu/cpufreq directory and put into the include/asm-i386 >directory. This was due to the fact that some of the moved files included >it, and some files that were not moved also included it. Instead of using >the #include "../../x86/" hack again, cd arch/i386/kernel/; ln -s ../../ia32 ia32; with obj-m += ia32/foo.c and #include "ia32/.../foo.h" a bit nicer IMO, but may not work as intended. > I just simply moved it to the global >i386 include directory. Only the arch/x86 will use the include/asm-i386 >change. But to make this change the move patches of the files that contain >this change also contain the changes to reference the change to locate this >file. > >With this change of having a single repo that holds both the x86_64 files >as well as the i386 code, it becomes obvious of what files are being shared. >This way we don't have to worry about someone changing a file in either >x86_64 or i386 and having it break the other arch, because they didn't >realize it was being shared. Jan -- - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Mar 14 2007 01:08, Steven Rostedt wrote: So I spent last night hacking up something to try to make a common ground for all code that is shared between x86_64 and i386. I called this arch/x86 Seems appropriate, but I really don't care what it's called. One thing about this name, is that typing arch/x86Tab doesn't tab complete x86_64 anymore. But if you can think of something better, I'd be happy to apply it. 80x86 8086 ia32 - I'd go with this for lack of more entries on this list. i486 i86pc - Solely listed for completes. No Solarisisms in Linux please :p ix86 x32- Blame Microsoft for introducing x64 x86_32 - Will interfere with your tabbing. So the moves are simply renames, with the slight exception of files that hold the speedstep-lib.h file. This file was moved from the arch/i386/kerne/cpu/cpufreq directory and put into the include/asm-i386 directory. This was due to the fact that some of the moved files included it, and some files that were not moved also included it. Instead of using the #include ../../x86/ hack again, cd arch/i386/kernel/; ln -s ../../ia32 ia32; with obj-m += ia32/foo.c and #include ia32/.../foo.h a bit nicer IMO, but may not work as intended. I just simply moved it to the global i386 include directory. Only the arch/x86 will use the include/asm-i386 change. But to make this change the move patches of the files that contain this change also contain the changes to reference the change to locate this file. With this change of having a single repo that holds both the x86_64 files as well as the i386 code, it becomes obvious of what files are being shared. This way we don't have to worry about someone changing a file in either x86_64 or i386 and having it break the other arch, because they didn't realize it was being shared. Jan -- - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
Steven Rostedt [EMAIL PROTECTED] writes: So I spent last night hacking up something to try to make a common ground for all code that is shared between x86_64 and i386. I called this arch/x86 NACK. I think the current ways work just fine. -Andi - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
Hi, I am newbie developing a routing application which needs three features; 1. if the fib lookup fails, my application needs to know about using preferably netlink, -- any direction to some sample code or files in the kernel??? 2. I need a counter recording the hits a fib entry is chosen for packet forwarding. Is it the function fn_trie_lookup in file /net/ipv4/fib_trie.c i need to modify and look into? -- ?? and therefore is it safe to add a int counter in fib_node w/o breaking the build?? 2. need to read the entire fib, i.e dump the fib to the user space, -- again any direction to some sample code etc.. i have seen a fn in /source/net/ipv4/fib_frontend.c , which looks like is supposed to dump the routes but not sure how to use it.. line no.581 int inet_dump_fib(struct sk_buff *skb, struct netlink_callback *cb) Also; fn_trie_dump() in /net/ipv4/fib_trie.c This could be an inappropriate list for the above questions, if it is request for some direction on the appropriate mailing list Appreciate any help. Thanking you, Sujay - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Wed, 2007-03-14 at 11:36 +0100, Andi Kleen wrote: Steven Rostedt [EMAIL PROTECTED] writes: So I spent last night hacking up something to try to make a common ground for all code that is shared between x86_64 and i386. I called this arch/x86 NACK. I think the current ways work just fine. Fine. The thing is others and I (and you) are working on getting paravirt_ops working for x86_64. There's a lot of overlap between i386 and x86_64. Right now the i386 is ahead of x86_64 and the code seems to be put more in the arch/i386 arch. So now we are going to introduce a new ../../i386 hack to get to a shared paravirt_shared.c(?). Or do we just continue on keeping the x86_64 as a separate entity, with a lot of duplicate code? I'm OK with either way. I just wanted to set a standard while it's still early enough to set one. -- Steve - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
* Andi Kleen [EMAIL PROTECTED] wrote: Steven Rostedt [EMAIL PROTECTED] writes: So I spent last night hacking up something to try to make a common ground for all code that is shared between x86_64 and i386. I called this arch/x86 NACK. I think the current ways work just fine. i agree. We've recently factored out quite a bit of common code between i386 and x86_64 recently: genirq, gtod/clocksource and clockevents. and that's how i think unification of architectures should be done: move code into kernel/* and drivers/*, _not_ into another architecture. That way all architectures benefit. Ingo - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
The thing is others and I (and you) are working on getting paravirt_ops working for x86_64. There's a lot of overlap between i386 and x86_64. Right now the i386 is ahead of x86_64 and the code seems to be put more in the arch/i386 arch. So now we are going to introduce a new ../../i386 hack to get to a shared paravirt_shared.c(?). Or do we What would you like exactly to share? just continue on keeping the x86_64 as a separate entity, with a lot of duplicate code? When it makes sense to share the code it can be shared. That is already done today. But the existing Makefile mechanisms work fine for that. Making sense: - There is actually a lot of shared code - The supported hardware is the same (remember x86-64 is only for modern x86 hardware, while i386 has a much longer legacy) - No ifdefs -Andi - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Wed, 2007-03-14 at 13:53 +0100, Ingo Molnar wrote: * Andi Kleen [EMAIL PROTECTED] wrote: Steven Rostedt [EMAIL PROTECTED] writes: So I spent last night hacking up something to try to make a common ground for all code that is shared between x86_64 and i386. I called this arch/x86 NACK. I think the current ways work just fine. i agree. We've recently factored out quite a bit of common code between i386 and x86_64 recently: genirq, gtod/clocksource and clockevents. But those are things that can mostly be shared across all archs. and that's how i think unification of architectures should be done: move code into kernel/* and drivers/*, _not_ into another architecture. That way all architectures benefit. I'm OK with it, although I did waste a lot of effort making those patches (But I speak better in C than in English, or any other verbal language, so it wasn't that bad). But if you look at the code that was merged, I'm not sure many other archs will benefit. How many archs use mtrr's? Perhaps these still can go into the drivers directory with a bit of work. Don't know, I'm not that close to that code to be sure, and don't have the time to find out ATM. But at least there needs to be a more common way to share files between the two archs. Having a file with just a single line of: #include ../../i386/kernel/mycommoncode.c is not that elegant. The make files are, perhaps, a bit better. Another thing that happens a lot with these shared funcs in these files, is finding them . Since make TAGS doesn't bother to check i386 when run with ARCH=x86_64. The first time I searched for early_printk while developing i386 took me an hour, since my search scripts don't check other archs (I've changed that since). I thought that the function was one of these macro created functions, and was non-arch specific (didn't look into arch). So, when creating new shared code, what's the proper way? -- Steve - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2
On Wed, 2007-03-14 at 14:05 +0100, Andi Kleen wrote: The thing is others and I (and you) are working on getting paravirt_ops working for x86_64. There's a lot of overlap between i386 and x86_64. Right now the i386 is ahead of x86_64 and the code seems to be put more in the arch/i386 arch. So now we are going to introduce a new ../../i386 hack to get to a shared paravirt_shared.c(?). Or do we What would you like exactly to share? I'm still working on that. Others have pressured us into consolidating as much usable code for i386 into x86_64. This may turn out not to be too feasible anyway. just continue on keeping the x86_64 as a separate entity, with a lot of duplicate code? When it makes sense to share the code it can be shared. That is already done today. But the existing Makefile mechanisms work fine for that. Making sense: - There is actually a lot of shared code - The supported hardware is the same (remember x86-64 is only for modern x86 hardware, while i386 has a much longer legacy) - No ifdefs Thanks, I'll keep this in mind. -- Steve - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
