Re: [PATCH v35 1/5] mm: support to get hints of free page blocks

2018-07-12 Thread Wei Wang
On 07/12/2018 07:49 PM, Michal Hocko wrote: On Thu 12-07-18 19:34:16, Wei Wang wrote: On 07/12/2018 04:13 PM, Michal Hocko wrote: On Thu 12-07-18 10:52:08, Wei Wang wrote: On 07/12/2018 10:30 AM, Linus Torvalds wrote: On Wed, Jul 11, 2018 at 7:17 PM Wei Wang wrote: Would it be better to rem

Re: [PATCH v35 1/5] mm: support to get hints of free page blocks

2018-07-12 Thread Michal Hocko
[Hmm this one somehow got stuck in my outgoing emails] On Wed 11-07-18 09:23:54, Linus Torvalds wrote: [...] > So I'm open to new interfaces. I just want those new interfaces to > make sense, and be low latency and simple for the VM to do. I'm > objecting to the incredibly baroque and heavy-weight

Re: [PATCH v35 1/5] mm: support to get hints of free page blocks

2018-07-12 Thread Michal Hocko
On Thu 12-07-18 19:34:16, Wei Wang wrote: > On 07/12/2018 04:13 PM, Michal Hocko wrote: > > On Thu 12-07-18 10:52:08, Wei Wang wrote: > > > On 07/12/2018 10:30 AM, Linus Torvalds wrote: > > > > On Wed, Jul 11, 2018 at 7:17 PM Wei Wang wrote: > > > > > Would it be better to remove __GFP_THISNODE? W

Re: [PATCH v35 1/5] mm: support to get hints of free page blocks

2018-07-12 Thread Wei Wang
On 07/12/2018 04:13 PM, Michal Hocko wrote: On Thu 12-07-18 10:52:08, Wei Wang wrote: On 07/12/2018 10:30 AM, Linus Torvalds wrote: On Wed, Jul 11, 2018 at 7:17 PM Wei Wang wrote: Would it be better to remove __GFP_THISNODE? We actually want to get all the guest free pages (from all the nodes

Re: [PATCH v35 1/5] mm: support to get hints of free page blocks

2018-07-12 Thread Michal Hocko
On Thu 12-07-18 10:52:08, Wei Wang wrote: > On 07/12/2018 10:30 AM, Linus Torvalds wrote: > > On Wed, Jul 11, 2018 at 7:17 PM Wei Wang wrote: > > > Would it be better to remove __GFP_THISNODE? We actually want to get all > > > the guest free pages (from all the nodes). > > Maybe. Or maybe it would

Re: [PATCH v35 1/5] mm: support to get hints of free page blocks

2018-07-11 Thread Wei Wang
On 07/12/2018 10:30 AM, Linus Torvalds wrote: On Wed, Jul 11, 2018 at 7:17 PM Wei Wang wrote: Would it be better to remove __GFP_THISNODE? We actually want to get all the guest free pages (from all the nodes). Maybe. Or maybe it would be better to have the memory balloon logic be per-node? May

Re: [PATCH v35 1/5] mm: support to get hints of free page blocks

2018-07-11 Thread Linus Torvalds
On Wed, Jul 11, 2018 at 7:17 PM Wei Wang wrote: > > Would it be better to remove __GFP_THISNODE? We actually want to get all > the guest free pages (from all the nodes). Maybe. Or maybe it would be better to have the memory balloon logic be per-node? Maybe you don't want to remove too much memory

Re: [PATCH v35 1/5] mm: support to get hints of free page blocks

2018-07-11 Thread Wei Wang
On 07/12/2018 12:23 AM, Linus Torvalds wrote: On Wed, Jul 11, 2018 at 2:21 AM Michal Hocko wrote: We already have an interface for that. alloc_pages(GFP_NOWAIT, MAX_ORDER -1). So why do we need any array based interface? That was actually my original argument in the original thread - that the

Re: [PATCH v35 1/5] mm: support to get hints of free page blocks

2018-07-11 Thread Michael S. Tsirkin
On Wed, Jul 11, 2018 at 01:09:49PM +0200, Michal Hocko wrote: > But let me note that I am not really convinced how this (or previous) > approach will really work in most workloads. We tend to cache heavily so > there is rarely any memory free. It might be that it's worth flushing the cache when VM

Re: [PATCH v35 1/5] mm: support to get hints of free page blocks

2018-07-11 Thread Linus Torvalds
On Wed, Jul 11, 2018 at 2:21 AM Michal Hocko wrote: > > We already have an interface for that. alloc_pages(GFP_NOWAIT, MAX_ORDER -1). > So why do we need any array based interface? That was actually my original argument in the original thread - that the only new interface people might want is one

Re: [PATCH v35 1/5] mm: support to get hints of free page blocks

2018-07-11 Thread Michal Hocko
On Wed 11-07-18 13:55:15, Wang, Wei W wrote: > On Wednesday, July 11, 2018 7:10 PM, Michal Hocko wrote: > > On Wed 11-07-18 18:52:45, Wei Wang wrote: > > > On 07/11/2018 05:21 PM, Michal Hocko wrote: > > > > On Tue 10-07-18 18:44:34, Linus Torvalds wrote: > > > > [...] > > > > > That was what I tri

RE: [PATCH v35 1/5] mm: support to get hints of free page blocks

2018-07-11 Thread Wang, Wei W
On Wednesday, July 11, 2018 7:10 PM, Michal Hocko wrote: > On Wed 11-07-18 18:52:45, Wei Wang wrote: > > On 07/11/2018 05:21 PM, Michal Hocko wrote: > > > On Tue 10-07-18 18:44:34, Linus Torvalds wrote: > > > [...] > > > > That was what I tried to encourage with actually removing the > > > > pages

Re: [PATCH v35 1/5] mm: support to get hints of free page blocks

2018-07-11 Thread Michal Hocko
On Wed 11-07-18 18:52:45, Wei Wang wrote: > On 07/11/2018 05:21 PM, Michal Hocko wrote: > > On Tue 10-07-18 18:44:34, Linus Torvalds wrote: > > [...] > > > That was what I tried to encourage with actually removing the pages > > > form the page list. That would be an _incremental_ interface. You can

Re: [PATCH v35 1/5] mm: support to get hints of free page blocks

2018-07-11 Thread Wei Wang
On 07/11/2018 05:21 PM, Michal Hocko wrote: On Tue 10-07-18 18:44:34, Linus Torvalds wrote: [...] That was what I tried to encourage with actually removing the pages form the page list. That would be an _incremental_ interface. You can remove MAX_ORDER-1 pages one by one (or a hundred at a time)

Re: [PATCH v35 1/5] mm: support to get hints of free page blocks

2018-07-11 Thread Michal Hocko
On Tue 10-07-18 18:44:34, Linus Torvalds wrote: [...] > That was what I tried to encourage with actually removing the pages > form the page list. That would be an _incremental_ interface. You can > remove MAX_ORDER-1 pages one by one (or a hundred at a time), and mark > them free for ballooning tha

Re: [PATCH v35 1/5] mm: support to get hints of free page blocks

2018-07-10 Thread Michael S. Tsirkin
On Wed, Jul 11, 2018 at 07:00:37AM +0300, Michael S. Tsirkin wrote: > On Tue, Jul 10, 2018 at 10:33:08AM -0700, Linus Torvalds wrote: > > NAK. > > > > On Tue, Jul 10, 2018 at 2:56 AM Wei Wang wrote: > > > > > > + > > > + buf_page = list_first_entry_or_null(pages, struct page, lru); > > > +

Re: [PATCH v35 1/5] mm: support to get hints of free page blocks

2018-07-10 Thread Michael S. Tsirkin
On Tue, Jul 10, 2018 at 10:33:08AM -0700, Linus Torvalds wrote: > NAK. > > On Tue, Jul 10, 2018 at 2:56 AM Wei Wang wrote: > > > > + > > + buf_page = list_first_entry_or_null(pages, struct page, lru); > > + if (!buf_page) > > + return -EINVAL; > > + buf = (__le64 *

Re: [PATCH v35 1/5] mm: support to get hints of free page blocks

2018-07-10 Thread Linus Torvalds
On Tue, Jul 10, 2018 at 6:24 PM Wei Wang wrote: > > We only get addresses of the "MAX_ORDER-1" blocks into the array. The > max size of the array that could be allocated by kmalloc is > KMALLOC_MAX_SIZE (i.e. 4MB on x86). With that max array, we could load > "4MB / sizeof(u64)" addresses of "MAX_O

Re: [PATCH v35 1/5] mm: support to get hints of free page blocks

2018-07-10 Thread Wei Wang
On 07/11/2018 01:33 AM, Linus Torvalds wrote: NAK. On Tue, Jul 10, 2018 at 2:56 AM Wei Wang wrote: + + buf_page = list_first_entry_or_null(pages, struct page, lru); + if (!buf_page) + return -EINVAL; + buf = (__le64 *)page_address(buf_page); Stop this garbage.

Re: [PATCH v35 1/5] mm: support to get hints of free page blocks

2018-07-10 Thread Linus Torvalds
NAK. On Tue, Jul 10, 2018 at 2:56 AM Wei Wang wrote: > > + > + buf_page = list_first_entry_or_null(pages, struct page, lru); > + if (!buf_page) > + return -EINVAL; > + buf = (__le64 *)page_address(buf_page); Stop this garbage. Why the hell would you pass in some

RE: [PATCH v35 1/5] mm: support to get hints of free page blocks

2018-07-10 Thread Wang, Wei W
On Tuesday, July 10, 2018 5:31 PM, Wang, Wei W wrote: > Subject: [PATCH v35 1/5] mm: support to get hints of free page blocks > > This patch adds support to get free page blocks from a free page list. > The physical addresses of the blocks are stored to a list of buffers passed >

[PATCH v35 1/5] mm: support to get hints of free page blocks

2018-07-10 Thread Wei Wang
This patch adds support to get free page blocks from a free page list. The physical addresses of the blocks are stored to a list of buffers passed from the caller. The obtained free page blocks are hints about free pages, because there is no guarantee that they are still on the free page list after