Re: [PATCH v4 00/22] x86, ACPI, numa: Parse numa info early
Hi HPA, Would you please tell me if this patch-set has been merged into any tree or branch ? If not merged, I'll rebased it to the latest kernel and resend it again. Hope the rebasing will help to push this patch-set. Thanks. :) On 05/22/2013 01:18 PM, H. Peter Anvin wrote: Sorry, just have been swamped since -rc1 came out. -hpa -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 00/22] x86, ACPI, numa: Parse numa info early
Hi HPA, Would you please tell me if this patch-set has been merged into any tree or branch ? If not merged, I'll rebased it to the latest kernel and resend it again. Hope the rebasing will help to push this patch-set. Thanks. :) On 05/22/2013 01:18 PM, H. Peter Anvin wrote: Sorry, just have been swamped since -rc1 came out. -hpa -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 00/22] x86, ACPI, numa: Parse numa info early
Sorry, just have been swamped since -rc1 came out. -hpa -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 00/22] x86, ACPI, numa: Parse numa info early
On 05/10/2013 02:24 AM, Yinghai Lu wrote: .. If you have any thinking of this patch-set, please let me know. Talked to HPA, and he will put my patchset into tip/x86/mm after v3.10-rc1. after that we can work on put pagetable on local node for hotadd path. Hi, It is Linux v3.10-rc2 now. But I didn't find this patch-set merged into tip/x86/mm. was it merged to somewhere else, or we have any other plan to push ? By the way, I have done some tests for this patch-set, and the test results have been sent. Please refer to: https://lkml.org/lkml/2013/4/30/45 Reviewed-by: Tang Chen Tested-by: Tang Chen Thanks. :) -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 00/22] x86, ACPI, numa: Parse numa info early
On 05/10/2013 02:24 AM, Yinghai Lu wrote: .. If you have any thinking of this patch-set, please let me know. Talked to HPA, and he will put my patchset into tip/x86/mm after v3.10-rc1. after that we can work on put pagetable on local node for hotadd path. Hi, It is Linux v3.10-rc2 now. But I didn't find this patch-set merged into tip/x86/mm. was it merged to somewhere else, or we have any other plan to push ? By the way, I have done some tests for this patch-set, and the test results have been sent. Please refer to: https://lkml.org/lkml/2013/4/30/45 Reviewed-by: Tang Chen tangc...@cn.fujitsu.com Tested-by: Tang Chen tangc...@cn.fujitsu.com Thanks. :) -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 00/22] x86, ACPI, numa: Parse numa info early
Sorry, just have been swamped since -rc1 came out. -hpa -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 00/22] x86, ACPI, numa: Parse numa info early
Hi Yinghai, How do you think of the following problem and solutions ? And can we not allocate pagetable to local node when MEMORY_HOTREMOVE is enabled for now, and do it again when the problem in hot-remove path is fixed ? Thanks. :) On 05/13/2013 10:59 AM, Tang Chen wrote: Hi Yinghai, On 05/10/2013 02:24 AM, Yinghai Lu wrote: So I suggest to separate the job into 2 parts: 1. Push Yinghai's patch1 ~ patch20, without putting pagetable in local node. And push my work to use SRAT to arrange ZONE_MOVABLE. In this case, we can enable memory hotplug in the kernel first. 2. Merge patch21 and patch22 into the fixing work I am doing now, and push them together when finished. no, no, no, please do not half-done work. Do it right, and Do it clean. I'm not saying I want to do it half-way. Putting pagetable in local node will make memory hot-remove patch unable to work. Before removing pages, the kernel first offlines pages. If the offline logic fails, the hot-remove cannot work. Since your patches have put node pagetable in local node at boot time, this memory cannot be offlined, furthermore, it cannot be hot-removed. The minimum unit of memory online/offline is block. And by default, one block contains one section, which by default is 128MB. So if parts of a block are pagetable, and the rest parts are movable memory, this block cannot be offlined. And as a result, it cannot be removed. In order to fix it, we have three solutions: 1. Reserve the whole block (128MB), making no user can use the rest parts of the block. And skip them when offlining memory. When all the other blocks are offlined, free the pagetable, and remove all the memory. But we may lose some memory for this purpose. 128MB is a little big to waste. 2. Migrate movable pages and keep this block online. Although the offline operation fails, it is OK to remove memory. But the offline operation will always fail. And generally speaking, there are a lot of reasons of offline failing, it is difficult to detect if it is OK to remove memory. 3. Migrate user pages and make this block offline, but the kernel can still use the pagetable in it. But this will change the semantics of "offline". I'm not sure if we can do it in this way. 4. Do not allocate pagetable to local node when CONFIG_MEMORY_HOTREMOVE is enabled. (I do suggest not to put pagetable in local node in memory hot-remove situation.) How do you think about these 4 solutions above ? I think I need some advices for this problem in community. Do you have any idea to fix this problem if we put pagetable in local node ? The memory hot-plug guys do want to use memory hot-remove. And I think for now, we use solution 4 above. When CONFIG_MEMORY_HOTREMOVE is enabled, do not allocate pagetable to local node. I'm not trying to do it half-way. When we fix this problem, we can allocate pagetable to local node again with CONFIG_MEMORY_HOTREMOVE enabled. Please do give some advices or feedback. If you have any thinking of this patch-set, please let me know. Talked to HPA, and he will put my patchset into tip/x86/mm after v3.10-rc1. after that we can work on put pagetable on local node for hotadd path. hot-add path is another problem. But I think the hot-remove path is more urgent now. Thanks. :) -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 00/22] x86, ACPI, numa: Parse numa info early
Hi Yinghai, How do you think of the following problem and solutions ? And can we not allocate pagetable to local node when MEMORY_HOTREMOVE is enabled for now, and do it again when the problem in hot-remove path is fixed ? Thanks. :) On 05/13/2013 10:59 AM, Tang Chen wrote: Hi Yinghai, On 05/10/2013 02:24 AM, Yinghai Lu wrote: So I suggest to separate the job into 2 parts: 1. Push Yinghai's patch1 ~ patch20, without putting pagetable in local node. And push my work to use SRAT to arrange ZONE_MOVABLE. In this case, we can enable memory hotplug in the kernel first. 2. Merge patch21 and patch22 into the fixing work I am doing now, and push them together when finished. no, no, no, please do not half-done work. Do it right, and Do it clean. I'm not saying I want to do it half-way. Putting pagetable in local node will make memory hot-remove patch unable to work. Before removing pages, the kernel first offlines pages. If the offline logic fails, the hot-remove cannot work. Since your patches have put node pagetable in local node at boot time, this memory cannot be offlined, furthermore, it cannot be hot-removed. The minimum unit of memory online/offline is block. And by default, one block contains one section, which by default is 128MB. So if parts of a block are pagetable, and the rest parts are movable memory, this block cannot be offlined. And as a result, it cannot be removed. In order to fix it, we have three solutions: 1. Reserve the whole block (128MB), making no user can use the rest parts of the block. And skip them when offlining memory. When all the other blocks are offlined, free the pagetable, and remove all the memory. But we may lose some memory for this purpose. 128MB is a little big to waste. 2. Migrate movable pages and keep this block online. Although the offline operation fails, it is OK to remove memory. But the offline operation will always fail. And generally speaking, there are a lot of reasons of offline failing, it is difficult to detect if it is OK to remove memory. 3. Migrate user pages and make this block offline, but the kernel can still use the pagetable in it. But this will change the semantics of offline. I'm not sure if we can do it in this way. 4. Do not allocate pagetable to local node when CONFIG_MEMORY_HOTREMOVE is enabled. (I do suggest not to put pagetable in local node in memory hot-remove situation.) How do you think about these 4 solutions above ? I think I need some advices for this problem in community. Do you have any idea to fix this problem if we put pagetable in local node ? The memory hot-plug guys do want to use memory hot-remove. And I think for now, we use solution 4 above. When CONFIG_MEMORY_HOTREMOVE is enabled, do not allocate pagetable to local node. I'm not trying to do it half-way. When we fix this problem, we can allocate pagetable to local node again with CONFIG_MEMORY_HOTREMOVE enabled. Please do give some advices or feedback. If you have any thinking of this patch-set, please let me know. Talked to HPA, and he will put my patchset into tip/x86/mm after v3.10-rc1. after that we can work on put pagetable on local node for hotadd path. hot-add path is another problem. But I think the hot-remove path is more urgent now. Thanks. :) -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 00/22] x86, ACPI, numa: Parse numa info early
Hi Yinghai, On 05/10/2013 02:24 AM, Yinghai Lu wrote: So I suggest to separate the job into 2 parts: 1. Push Yinghai's patch1 ~ patch20, without putting pagetable in local node. And push my work to use SRAT to arrange ZONE_MOVABLE. In this case, we can enable memory hotplug in the kernel first. 2. Merge patch21 and patch22 into the fixing work I am doing now, and push them together when finished. no, no, no, please do not half-done work. Do it right, and Do it clean. I'm not saying I want to do it half-way. Putting pagetable in local node will make memory hot-remove patch unable to work. Before removing pages, the kernel first offlines pages. If the offline logic fails, the hot-remove cannot work. Since your patches have put node pagetable in local node at boot time, this memory cannot be offlined, furthermore, it cannot be hot-removed. The minimum unit of memory online/offline is block. And by default, one block contains one section, which by default is 128MB. So if parts of a block are pagetable, and the rest parts are movable memory, this block cannot be offlined. And as a result, it cannot be removed. In order to fix it, we have three solutions: 1. Reserve the whole block (128MB), making no user can use the rest parts of the block. And skip them when offlining memory. When all the other blocks are offlined, free the pagetable, and remove all the memory. But we may lose some memory for this purpose. 128MB is a little big to waste. 2. Migrate movable pages and keep this block online. Although the offline operation fails, it is OK to remove memory. But the offline operation will always fail. And generally speaking, there are a lot of reasons of offline failing, it is difficult to detect if it is OK to remove memory. 3. Migrate user pages and make this block offline, but the kernel can still use the pagetable in it. But this will change the semantics of "offline". I'm not sure if we can do it in this way. 4. Do not allocate pagetable to local node when CONFIG_MEMORY_HOTREMOVE is enabled. (I do suggest not to put pagetable in local node in memory hot-remove situation.) How do you think about these 4 solutions above ? I think I need some advices for this problem in community. Do you have any idea to fix this problem if we put pagetable in local node ? The memory hot-plug guys do want to use memory hot-remove. And I think for now, we use solution 4 above. When CONFIG_MEMORY_HOTREMOVE is enabled, do not allocate pagetable to local node. I'm not trying to do it half-way. When we fix this problem, we can allocate pagetable to local node again with CONFIG_MEMORY_HOTREMOVE enabled. Please do give some advices or feedback. If you have any thinking of this patch-set, please let me know. Talked to HPA, and he will put my patchset into tip/x86/mm after v3.10-rc1. after that we can work on put pagetable on local node for hotadd path. hot-add path is another problem. But I think the hot-remove path is more urgent now. Thanks. :) -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 00/22] x86, ACPI, numa: Parse numa info early
Hi Yinghai, On 05/10/2013 02:24 AM, Yinghai Lu wrote: So I suggest to separate the job into 2 parts: 1. Push Yinghai's patch1 ~ patch20, without putting pagetable in local node. And push my work to use SRAT to arrange ZONE_MOVABLE. In this case, we can enable memory hotplug in the kernel first. 2. Merge patch21 and patch22 into the fixing work I am doing now, and push them together when finished. no, no, no, please do not half-done work. Do it right, and Do it clean. I'm not saying I want to do it half-way. Putting pagetable in local node will make memory hot-remove patch unable to work. Before removing pages, the kernel first offlines pages. If the offline logic fails, the hot-remove cannot work. Since your patches have put node pagetable in local node at boot time, this memory cannot be offlined, furthermore, it cannot be hot-removed. The minimum unit of memory online/offline is block. And by default, one block contains one section, which by default is 128MB. So if parts of a block are pagetable, and the rest parts are movable memory, this block cannot be offlined. And as a result, it cannot be removed. In order to fix it, we have three solutions: 1. Reserve the whole block (128MB), making no user can use the rest parts of the block. And skip them when offlining memory. When all the other blocks are offlined, free the pagetable, and remove all the memory. But we may lose some memory for this purpose. 128MB is a little big to waste. 2. Migrate movable pages and keep this block online. Although the offline operation fails, it is OK to remove memory. But the offline operation will always fail. And generally speaking, there are a lot of reasons of offline failing, it is difficult to detect if it is OK to remove memory. 3. Migrate user pages and make this block offline, but the kernel can still use the pagetable in it. But this will change the semantics of offline. I'm not sure if we can do it in this way. 4. Do not allocate pagetable to local node when CONFIG_MEMORY_HOTREMOVE is enabled. (I do suggest not to put pagetable in local node in memory hot-remove situation.) How do you think about these 4 solutions above ? I think I need some advices for this problem in community. Do you have any idea to fix this problem if we put pagetable in local node ? The memory hot-plug guys do want to use memory hot-remove. And I think for now, we use solution 4 above. When CONFIG_MEMORY_HOTREMOVE is enabled, do not allocate pagetable to local node. I'm not trying to do it half-way. When we fix this problem, we can allocate pagetable to local node again with CONFIG_MEMORY_HOTREMOVE enabled. Please do give some advices or feedback. If you have any thinking of this patch-set, please let me know. Talked to HPA, and he will put my patchset into tip/x86/mm after v3.10-rc1. after that we can work on put pagetable on local node for hotadd path. hot-add path is another problem. But I think the hot-remove path is more urgent now. Thanks. :) -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 00/22] x86, ACPI, numa: Parse numa info early
On Thu, May 9, 2013 at 1:54 AM, Tang Chen wrote: > Hi Yinghai, > > > On 04/30/2013 03:21 PM, Tang Chen wrote: >> >> So I suggest to separate the job into 2 parts: >> 1. Push Yinghai's patch1 ~ patch20, without putting pagetable in local >> node. >> And push my work to use SRAT to arrange ZONE_MOVABLE. >> In this case, we can enable memory hotplug in the kernel first. >> 2. Merge patch21 and patch22 into the fixing work I am doing now, and push >> them >> together when finished. >> > > It has been a long time since this mail and there was no response. I do > think I > should move on to push this patch-set. So if you don't mind, I'll rebase and > push "parse SRAT earlier" part of this patch-set first. > > Since putting pagetable in local node will destroy memory hot-remove logic > for now, > I will drop "put pagetable in local node" parts now, and merge this part > into > the hot-add and hot-remove fix work. no, no, no, please do not half-done work. Do it right, and Do it clean. > > If you have any thinking of this patch-set, please let me know. Talked to HPA, and he will put my patchset into tip/x86/mm after v3.10-rc1. after that we can work on put pagetable on local node for hotadd path. Thanks Yinghai -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 00/22] x86, ACPI, numa: Parse numa info early
Hi Yinghai, On 04/30/2013 03:21 PM, Tang Chen wrote: So I suggest to separate the job into 2 parts: 1. Push Yinghai's patch1 ~ patch20, without putting pagetable in local node. And push my work to use SRAT to arrange ZONE_MOVABLE. In this case, we can enable memory hotplug in the kernel first. 2. Merge patch21 and patch22 into the fixing work I am doing now, and push them together when finished. It has been a long time since this mail and there was no response. I do think I should move on to push this patch-set. So if you don't mind, I'll rebase and push "parse SRAT earlier" part of this patch-set first. Since putting pagetable in local node will destroy memory hot-remove logic for now, I will drop "put pagetable in local node" parts now, and merge this part into the hot-add and hot-remove fix work. If you have any thinking of this patch-set, please let me know. Thanks. :) -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 00/22] x86, ACPI, numa: Parse numa info early
Hi Yinghai, On 04/30/2013 03:21 PM, Tang Chen wrote: So I suggest to separate the job into 2 parts: 1. Push Yinghai's patch1 ~ patch20, without putting pagetable in local node. And push my work to use SRAT to arrange ZONE_MOVABLE. In this case, we can enable memory hotplug in the kernel first. 2. Merge patch21 and patch22 into the fixing work I am doing now, and push them together when finished. It has been a long time since this mail and there was no response. I do think I should move on to push this patch-set. So if you don't mind, I'll rebase and push parse SRAT earlier part of this patch-set first. Since putting pagetable in local node will destroy memory hot-remove logic for now, I will drop put pagetable in local node parts now, and merge this part into the hot-add and hot-remove fix work. If you have any thinking of this patch-set, please let me know. Thanks. :) -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 00/22] x86, ACPI, numa: Parse numa info early
On Thu, May 9, 2013 at 1:54 AM, Tang Chen tangc...@cn.fujitsu.com wrote: Hi Yinghai, On 04/30/2013 03:21 PM, Tang Chen wrote: So I suggest to separate the job into 2 parts: 1. Push Yinghai's patch1 ~ patch20, without putting pagetable in local node. And push my work to use SRAT to arrange ZONE_MOVABLE. In this case, we can enable memory hotplug in the kernel first. 2. Merge patch21 and patch22 into the fixing work I am doing now, and push them together when finished. It has been a long time since this mail and there was no response. I do think I should move on to push this patch-set. So if you don't mind, I'll rebase and push parse SRAT earlier part of this patch-set first. Since putting pagetable in local node will destroy memory hot-remove logic for now, I will drop put pagetable in local node parts now, and merge this part into the hot-add and hot-remove fix work. no, no, no, please do not half-done work. Do it right, and Do it clean. If you have any thinking of this patch-set, please let me know. Talked to HPA, and he will put my patchset into tip/x86/mm after v3.10-rc1. after that we can work on put pagetable on local node for hotadd path. Thanks Yinghai -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 00/22] x86, ACPI, numa: Parse numa info early
Hi all, Could anyone give some suggestions to this patch-set ? Thanks. On 04/30/2013 03:21 PM, Tang Chen wrote: Hi Yinghai, all, I've tested this patch-set with my following patch-set: [PATCH v1 00/12] Arrange hotpluggable memory in SRAT as ZONE_MOVABLE. https://lkml.org/lkml/2013/4/19/94 Using ACPI table override, I overrided SRAT on my box like this: [ 0.00] SRAT: Node 0 PXM 0 [mem 0x-0x7fff] [ 0.00] SRAT: Node 0 PXM 0 [mem 0x1-0x307ff] [ 0.00] SRAT: Node 1 PXM 2 [mem 0x30800-0x583ff] Hot Pluggable [ 0.00] SRAT: Node 2 PXM 3 [mem 0x58400-0x7] Hot Pluggable We had 3 nodes, node0 was not hotpluggable, and node1 and node2 were hotpluggable. And memblock reserved pagetable pages (with flag 0x1) in local nodes. .. [ 0.00] reserved[0xb] [0x0307ff-0x0307ff1fff], 0x2000 bytes flags: 0x0 [ 0.00] reserved[0xc] [0x0307ff2000-0x0307ff], 0xe000 bytes on node 0 flags: 0x1 [ 0.00] reserved[0xd] [0x0583ff7000-0x0583ff], 0x9000 bytes on node 1 flags: 0x1 [ 0.00] reserved[0xe] [0x079000-0x07], 0x7000 bytes on node 2 flags: 0x1 And after some bug fix, memblock can also reserve hotpluggable memory with flag 0x2. .. [ 0.00] reserved[0xb] [0x0307ff-0x0307ff1fff], 0x2000 bytes flags: 0x0 [ 0.00] reserved[0xc] [0x0307ff2000-0x0307ff], 0xe000 bytes on node 0 flags: 0x1 [ 0.00] reserved[0xd] [0x030800-0x0583ff6fff], 0x27bff7000 bytes on node 1 flags: 0x2 [ 0.00] reserved[0xe] [0x0583ff7000-0x0583ff], 0x9000 bytes on node 1 flags: 0x1 [ 0.00] reserved[0xf] [0x058400-0x077fff], 0x27bff8000 bytes on node 2 flags: 0x2 [ 0.00] reserved[0x10] [0x078000-0x07], 0x8000 bytes on node 2 flags: 0x1 And free it to buddy system when memory initialization finished. So the results: 1. We can parse SRAT earlier correctly. 2. We can override tables correctly. 3. We can put pagetable pages in local node. 4. We can prevent memblock from allocating hotpluggable memory. 5. We can arrange ZONE_MOVABLE using SRAT info. Known problems: When we put pagetable pages in local node, memory hot-remove logic won't work. I'm fixing it now. We need to fix the following: 1. Improve hot-remove to support freeing local node pagetable pages. 2. Improve hot-add to support putting hot-added pagetable pages in local node. 3. Do the same to vmemmap and page_cgrop pages. So I suggest to separate the job into 2 parts: 1. Push Yinghai's patch1 ~ patch20, without putting pagetable in local node. And push my work to use SRAT to arrange ZONE_MOVABLE. In this case, we can enable memory hotplug in the kernel first. 2. Merge patch21 and patch22 into the fixing work I am doing now, and push them together when finished. How do you think ? Reviewed-by: Tang Chen Tested-by: Tang Chen Thanks. :) -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 00/22] x86, ACPI, numa: Parse numa info early
Hi all, Could anyone give some suggestions to this patch-set ? Thanks. On 04/30/2013 03:21 PM, Tang Chen wrote: Hi Yinghai, all, I've tested this patch-set with my following patch-set: [PATCH v1 00/12] Arrange hotpluggable memory in SRAT as ZONE_MOVABLE. https://lkml.org/lkml/2013/4/19/94 Using ACPI table override, I overrided SRAT on my box like this: [ 0.00] SRAT: Node 0 PXM 0 [mem 0x-0x7fff] [ 0.00] SRAT: Node 0 PXM 0 [mem 0x1-0x307ff] [ 0.00] SRAT: Node 1 PXM 2 [mem 0x30800-0x583ff] Hot Pluggable [ 0.00] SRAT: Node 2 PXM 3 [mem 0x58400-0x7] Hot Pluggable We had 3 nodes, node0 was not hotpluggable, and node1 and node2 were hotpluggable. And memblock reserved pagetable pages (with flag 0x1) in local nodes. .. [ 0.00] reserved[0xb] [0x0307ff-0x0307ff1fff], 0x2000 bytes flags: 0x0 [ 0.00] reserved[0xc] [0x0307ff2000-0x0307ff], 0xe000 bytes on node 0 flags: 0x1 [ 0.00] reserved[0xd] [0x0583ff7000-0x0583ff], 0x9000 bytes on node 1 flags: 0x1 [ 0.00] reserved[0xe] [0x079000-0x07], 0x7000 bytes on node 2 flags: 0x1 And after some bug fix, memblock can also reserve hotpluggable memory with flag 0x2. .. [ 0.00] reserved[0xb] [0x0307ff-0x0307ff1fff], 0x2000 bytes flags: 0x0 [ 0.00] reserved[0xc] [0x0307ff2000-0x0307ff], 0xe000 bytes on node 0 flags: 0x1 [ 0.00] reserved[0xd] [0x030800-0x0583ff6fff], 0x27bff7000 bytes on node 1 flags: 0x2 [ 0.00] reserved[0xe] [0x0583ff7000-0x0583ff], 0x9000 bytes on node 1 flags: 0x1 [ 0.00] reserved[0xf] [0x058400-0x077fff], 0x27bff8000 bytes on node 2 flags: 0x2 [ 0.00] reserved[0x10] [0x078000-0x07], 0x8000 bytes on node 2 flags: 0x1 And free it to buddy system when memory initialization finished. So the results: 1. We can parse SRAT earlier correctly. 2. We can override tables correctly. 3. We can put pagetable pages in local node. 4. We can prevent memblock from allocating hotpluggable memory. 5. We can arrange ZONE_MOVABLE using SRAT info. Known problems: When we put pagetable pages in local node, memory hot-remove logic won't work. I'm fixing it now. We need to fix the following: 1. Improve hot-remove to support freeing local node pagetable pages. 2. Improve hot-add to support putting hot-added pagetable pages in local node. 3. Do the same to vmemmap and page_cgrop pages. So I suggest to separate the job into 2 parts: 1. Push Yinghai's patch1 ~ patch20, without putting pagetable in local node. And push my work to use SRAT to arrange ZONE_MOVABLE. In this case, we can enable memory hotplug in the kernel first. 2. Merge patch21 and patch22 into the fixing work I am doing now, and push them together when finished. How do you think ? Reviewed-by: Tang Chen tangc...@cn.fujitsu.com Tested-by: Tang Chen tangc...@cn.fujitsu.com Thanks. :) -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 00/22] x86, ACPI, numa: Parse numa info early
Hi Yinghai, all, I've tested this patch-set with my following patch-set: [PATCH v1 00/12] Arrange hotpluggable memory in SRAT as ZONE_MOVABLE. https://lkml.org/lkml/2013/4/19/94 Using ACPI table override, I overrided SRAT on my box like this: [0.00] SRAT: Node 0 PXM 0 [mem 0x-0x7fff] [0.00] SRAT: Node 0 PXM 0 [mem 0x1-0x307ff] [0.00] SRAT: Node 1 PXM 2 [mem 0x30800-0x583ff] Hot Pluggable [0.00] SRAT: Node 2 PXM 3 [mem 0x58400-0x7] Hot Pluggable We had 3 nodes, node0 was not hotpluggable, and node1 and node2 were hotpluggable. And memblock reserved pagetable pages (with flag 0x1) in local nodes. .. [0.00] reserved[0xb] [0x0307ff-0x0307ff1fff], 0x2000 bytes flags: 0x0 [0.00] reserved[0xc] [0x0307ff2000-0x0307ff], 0xe000 bytes on node 0 flags: 0x1 [0.00] reserved[0xd] [0x0583ff7000-0x0583ff], 0x9000 bytes on node 1 flags: 0x1 [0.00] reserved[0xe] [0x079000-0x07], 0x7000 bytes on node 2 flags: 0x1 And after some bug fix, memblock can also reserve hotpluggable memory with flag 0x2. .. [0.00] reserved[0xb] [0x0307ff-0x0307ff1fff], 0x2000 bytes flags: 0x0 [0.00] reserved[0xc] [0x0307ff2000-0x0307ff], 0xe000 bytes on node 0 flags: 0x1 [0.00] reserved[0xd] [0x030800-0x0583ff6fff], 0x27bff7000 bytes on node 1 flags: 0x2 [0.00] reserved[0xe] [0x0583ff7000-0x0583ff], 0x9000 bytes on node 1 flags: 0x1 [0.00] reserved[0xf] [0x058400-0x077fff], 0x27bff8000 bytes on node 2 flags: 0x2 [0.00] reserved[0x10] [0x078000-0x07], 0x8000 bytes on node 2 flags: 0x1 And free it to buddy system when memory initialization finished. So the results: 1. We can parse SRAT earlier correctly. 2. We can override tables correctly. 3. We can put pagetable pages in local node. 4. We can prevent memblock from allocating hotpluggable memory. 5. We can arrange ZONE_MOVABLE using SRAT info. Known problems: When we put pagetable pages in local node, memory hot-remove logic won't work. I'm fixing it now. We need to fix the following: 1. Improve hot-remove to support freeing local node pagetable pages. 2. Improve hot-add to support putting hot-added pagetable pages in local node. 3. Do the same to vmemmap and page_cgrop pages. So I suggest to separate the job into 2 parts: 1. Push Yinghai's patch1 ~ patch20, without putting pagetable in local node. And push my work to use SRAT to arrange ZONE_MOVABLE. In this case, we can enable memory hotplug in the kernel first. 2. Merge patch21 and patch22 into the fixing work I am doing now, and push them together when finished. How do you think ? Reviewed-by: Tang Chen Tested-by: Tang Chen Thanks. :) -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 00/22] x86, ACPI, numa: Parse numa info early
Hi Yinghai, all, I've tested this patch-set with my following patch-set: [PATCH v1 00/12] Arrange hotpluggable memory in SRAT as ZONE_MOVABLE. https://lkml.org/lkml/2013/4/19/94 Using ACPI table override, I overrided SRAT on my box like this: [0.00] SRAT: Node 0 PXM 0 [mem 0x-0x7fff] [0.00] SRAT: Node 0 PXM 0 [mem 0x1-0x307ff] [0.00] SRAT: Node 1 PXM 2 [mem 0x30800-0x583ff] Hot Pluggable [0.00] SRAT: Node 2 PXM 3 [mem 0x58400-0x7] Hot Pluggable We had 3 nodes, node0 was not hotpluggable, and node1 and node2 were hotpluggable. And memblock reserved pagetable pages (with flag 0x1) in local nodes. .. [0.00] reserved[0xb] [0x0307ff-0x0307ff1fff], 0x2000 bytes flags: 0x0 [0.00] reserved[0xc] [0x0307ff2000-0x0307ff], 0xe000 bytes on node 0 flags: 0x1 [0.00] reserved[0xd] [0x0583ff7000-0x0583ff], 0x9000 bytes on node 1 flags: 0x1 [0.00] reserved[0xe] [0x079000-0x07], 0x7000 bytes on node 2 flags: 0x1 And after some bug fix, memblock can also reserve hotpluggable memory with flag 0x2. .. [0.00] reserved[0xb] [0x0307ff-0x0307ff1fff], 0x2000 bytes flags: 0x0 [0.00] reserved[0xc] [0x0307ff2000-0x0307ff], 0xe000 bytes on node 0 flags: 0x1 [0.00] reserved[0xd] [0x030800-0x0583ff6fff], 0x27bff7000 bytes on node 1 flags: 0x2 [0.00] reserved[0xe] [0x0583ff7000-0x0583ff], 0x9000 bytes on node 1 flags: 0x1 [0.00] reserved[0xf] [0x058400-0x077fff], 0x27bff8000 bytes on node 2 flags: 0x2 [0.00] reserved[0x10] [0x078000-0x07], 0x8000 bytes on node 2 flags: 0x1 And free it to buddy system when memory initialization finished. So the results: 1. We can parse SRAT earlier correctly. 2. We can override tables correctly. 3. We can put pagetable pages in local node. 4. We can prevent memblock from allocating hotpluggable memory. 5. We can arrange ZONE_MOVABLE using SRAT info. Known problems: When we put pagetable pages in local node, memory hot-remove logic won't work. I'm fixing it now. We need to fix the following: 1. Improve hot-remove to support freeing local node pagetable pages. 2. Improve hot-add to support putting hot-added pagetable pages in local node. 3. Do the same to vmemmap and page_cgrop pages. So I suggest to separate the job into 2 parts: 1. Push Yinghai's patch1 ~ patch20, without putting pagetable in local node. And push my work to use SRAT to arrange ZONE_MOVABLE. In this case, we can enable memory hotplug in the kernel first. 2. Merge patch21 and patch22 into the fixing work I am doing now, and push them together when finished. How do you think ? Reviewed-by: Tang Chen tangc...@cn.fujitsu.com Tested-by: Tang Chen tangc...@cn.fujitsu.com Thanks. :) -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 00/22] x86, ACPI, numa: Parse numa info early
Hi Yinghai, It has been a long time since this patch-set was sent. I think we need to do something to push it. In my understanding, this patch-set did 2 things. 1. Parse numa info earlier, some improvements for ACPI_INITRD_TABLE_OVERRIDE. (patch1 ~ patch20) 2. Allocate pagetable in local node at boot time. (patch21 ~ patch22) As you know, the current implement of memory hot-remove is not based on putting pagetable in local node. If we put pagetable in local node at boot time, the memory hot-remove won't be able to work as before. I agree that this should be fixed. But we have the following two reasons to push "Parse numa info earlier" part first, and improve the performance later. 1. patch21 and patch22 only affect the performance, not the functionality. I think we can make memory hot-remove work in the kernel, and than improve the performance. 2. Besides putting pagetable in local node at boot time, there are many other things need to do. I'm working on improving hot-add code to allocate pagetable and vmemmap in local node, and improving hot-remove code to support freeing this kind of memory. So in order to push this patch-set and memory hot-remove functionality, shall we divide this patch-set into 2 steps: 1. Push patch1 ~ patch20, and I'll push the remaining memory hot-remove work together. 2. Merge your "putting pagetable in local node" work with the performance improvement work I'm doing, and improve the performance. How do you think ? BTW, I'm testing your patch-set, and will give a result next week. I can also help to rebase it if you like. Thanks. :) -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 00/22] x86, ACPI, numa: Parse numa info early
Hi Yinghai, It has been a long time since this patch-set was sent. I think we need to do something to push it. In my understanding, this patch-set did 2 things. 1. Parse numa info earlier, some improvements for ACPI_INITRD_TABLE_OVERRIDE. (patch1 ~ patch20) 2. Allocate pagetable in local node at boot time. (patch21 ~ patch22) As you know, the current implement of memory hot-remove is not based on putting pagetable in local node. If we put pagetable in local node at boot time, the memory hot-remove won't be able to work as before. I agree that this should be fixed. But we have the following two reasons to push Parse numa info earlier part first, and improve the performance later. 1. patch21 and patch22 only affect the performance, not the functionality. I think we can make memory hot-remove work in the kernel, and than improve the performance. 2. Besides putting pagetable in local node at boot time, there are many other things need to do. I'm working on improving hot-add code to allocate pagetable and vmemmap in local node, and improving hot-remove code to support freeing this kind of memory. So in order to push this patch-set and memory hot-remove functionality, shall we divide this patch-set into 2 steps: 1. Push patch1 ~ patch20, and I'll push the remaining memory hot-remove work together. 2. Merge your putting pagetable in local node work with the performance improvement work I'm doing, and improve the performance. How do you think ? BTW, I'm testing your patch-set, and will give a result next week. I can also help to rebase it if you like. Thanks. :) -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
