Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Michael S. Tsirkin
On Wed, Aug 22, 2012 at 11:19:04PM -0300, Rafael Aquini wrote: On Wed, Aug 22, 2012 at 12:33:17PM +0300, Michael S. Tsirkin wrote: Hmm, so this will busy wait which is unelegant. We need some event IMO. No, it does not busy wait. leak_balloon() is mutual exclusive with migration steps, so

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 01:01:07PM +0300, Michael S. Tsirkin wrote: So, when remove_common() calls leak_balloon() looping on vb-num_pages, that won't become a tight loop. The scheme was apparently working before this series, and it will remain working after it. It seems that before

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Michael S. Tsirkin
On Thu, Aug 23, 2012 at 09:13:39AM -0300, Rafael Aquini wrote: On Thu, Aug 23, 2012 at 01:01:07PM +0300, Michael S. Tsirkin wrote: So, when remove_common() calls leak_balloon() looping on vb-num_pages, that won't become a tight loop. The scheme was apparently working before this series,

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 03:34:32PM +0300, Michael S. Tsirkin wrote: So, nothing has changed here. Yes, your patch does change things: leak_balloon now might return without freeing any pages. In that case we will not be making any progress, and just spin, pinning CPU. That's a transitory

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Michael S. Tsirkin
On Thu, Aug 23, 2012 at 10:06:07AM -0300, Rafael Aquini wrote: On Thu, Aug 23, 2012 at 03:34:32PM +0300, Michael S. Tsirkin wrote: So, nothing has changed here. Yes, your patch does change things: leak_balloon now might return without freeing any pages. In that case we will not be

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 04:53:29PM +0300, Michael S. Tsirkin wrote: On Thu, Aug 23, 2012 at 10:06:07AM -0300, Rafael Aquini wrote: On Thu, Aug 23, 2012 at 03:34:32PM +0300, Michael S. Tsirkin wrote: So, nothing has changed here. Yes, your patch does change things: leak_balloon now

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Michael S. Tsirkin
On Thu, Aug 23, 2012 at 12:21:29PM -0300, Rafael Aquini wrote: On Thu, Aug 23, 2012 at 04:53:29PM +0300, Michael S. Tsirkin wrote: On Thu, Aug 23, 2012 at 10:06:07AM -0300, Rafael Aquini wrote: On Thu, Aug 23, 2012 at 03:34:32PM +0300, Michael S. Tsirkin wrote: So, nothing has changed

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rik van Riel
On 08/23/2012 11:54 AM, Michael S. Tsirkin wrote: On Thu, Aug 23, 2012 at 12:21:29PM -0300, Rafael Aquini wrote: On Thu, Aug 23, 2012 at 04:53:29PM +0300, Michael S. Tsirkin wrote: On Thu, Aug 23, 2012 at 10:06:07AM -0300, Rafael Aquini wrote: On Thu, Aug 23, 2012 at 03:34:32PM +0300, Michael

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 12:03:15PM -0400, Rik van Riel wrote: Not longer - apparently forever unless user resend the leak command. It's wrong - it should 1. not tell host if nothing was done 2. after migration finished leak and tell host Agreed. If the balloon is told to leak N pages,

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Michael S. Tsirkin
On Thu, Aug 23, 2012 at 01:06:48PM -0300, Rafael Aquini wrote: On Thu, Aug 23, 2012 at 12:03:15PM -0400, Rik van Riel wrote: Not longer - apparently forever unless user resend the leak command. It's wrong - it should 1. not tell host if nothing was done 2. after migration finished leak

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Michael S. Tsirkin
On Thu, Aug 23, 2012 at 04:53:28PM +0300, Michael S. Tsirkin wrote: Basically it was very simple: we assumed page-lru was never touched for an allocated page, so it's safe to use it for internal book-keeping by the driver. Now, this is not the case anymore, you add some logic in mm/ that

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 07:25:05PM +0300, Michael S. Tsirkin wrote: On Thu, Aug 23, 2012 at 04:53:28PM +0300, Michael S. Tsirkin wrote: Basically it was very simple: we assumed page-lru was never touched for an allocated page, so it's safe to use it for internal book-keeping by the driver.

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rik van Riel
On 08/23/2012 01:28 PM, Rafael Aquini wrote: What about keep a conter at virtio_balloon structure on how much pages are isolated from balloon's list and check it at leak time? if the counter gets 0 than we can safely put leak_balloon() to wait until balloon page list gets completely refilled.

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Michael S. Tsirkin
On Thu, Aug 23, 2012 at 02:28:45PM -0300, Rafael Aquini wrote: On Thu, Aug 23, 2012 at 07:25:05PM +0300, Michael S. Tsirkin wrote: On Thu, Aug 23, 2012 at 04:53:28PM +0300, Michael S. Tsirkin wrote: Basically it was very simple: we assumed page-lru was never touched for an allocated page,

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Fri, Aug 24, 2012 at 02:36:16AM +0300, Michael S. Tsirkin wrote: On Thu, Aug 23, 2012 at 02:28:45PM -0300, Rafael Aquini wrote: On Thu, Aug 23, 2012 at 07:25:05PM +0300, Michael S. Tsirkin wrote: On Thu, Aug 23, 2012 at 04:53:28PM +0300, Michael S. Tsirkin wrote: Basically it was very

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Fri, Aug 24, 2012 at 02:36:16AM +0300, Michael S. Tsirkin wrote: I would wake it each time after adding a page, then it can stop waiting when it leaks enough. But again, it's cleaner to just keep tracking all pages, let mm hang on to them by keeping a reference. Here is a rough idea on

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 09:33:53PM -0300, Rafael Aquini wrote: On Fri, Aug 24, 2012 at 02:36:16AM +0300, Michael S. Tsirkin wrote: I would wake it each time after adding a page, then it can stop waiting when it leaks enough. But again, it's cleaner to just keep tracking all pages, let mm

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 09:38:48PM -0300, Rafael Aquini wrote: On Thu, Aug 23, 2012 at 09:33:53PM -0300, Rafael Aquini wrote: On Fri, Aug 24, 2012 at 02:36:16AM +0300, Michael S. Tsirkin wrote: I would wake it each time after adding a page, then it can stop waiting when it leaks enough.

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rik van Riel
On 08/23/2012 07:36 PM, Michael S. Tsirkin wrote: --- virtio-balloon: replace page-lru list with page-private. The point is to free up page-lru for use by compaction. Warning: completely untested, will provide tested version if we agree on this direction. A singly linked list is not going

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-22 Thread Michael S. Tsirkin
On Tue, Aug 21, 2012 at 10:19:31PM -0300, Rafael Aquini wrote: On Wed, Aug 22, 2012 at 03:07:41AM +0300, Michael S. Tsirkin wrote: On Tue, Aug 21, 2012 at 05:45:56PM -0300, Rafael Aquini wrote: On Tue, Aug 21, 2012 at 10:30:31PM +0300, Michael S. Tsirkin wrote: On Tue, Aug 21, 2012 at

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-22 Thread Rafael Aquini
On Wed, Aug 22, 2012 at 12:33:17PM +0300, Michael S. Tsirkin wrote: Hmm, so this will busy wait which is unelegant. We need some event IMO. No, it does not busy wait. leak_balloon() is mutual exclusive with migration steps, so for the case we have one racing against the other, we really want

[PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
Memory fragmentation introduced by ballooning might reduce significantly the number of 2MB contiguous memory blocks that can be used within a guest, thus imposing performance penalties associated with the reduced number of transparent huge pages that could be used by the guest workload. This

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Michael S. Tsirkin
On Tue, Aug 21, 2012 at 09:47:44AM -0300, Rafael Aquini wrote: Memory fragmentation introduced by ballooning might reduce significantly the number of 2MB contiguous memory blocks that can be used within a guest, thus imposing performance penalties associated with the reduced number of

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Michael S. Tsirkin
On Tue, Aug 21, 2012 at 04:52:23PM +0300, Michael S. Tsirkin wrote: On Tue, Aug 21, 2012 at 09:47:44AM -0300, Rafael Aquini wrote: Memory fragmentation introduced by ballooning might reduce significantly the number of 2MB contiguous memory blocks that can be used within a guest, thus

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Michael S. Tsirkin
On Tue, Aug 21, 2012 at 05:16:06PM +0200, Peter Zijlstra wrote: On Tue, 2012-08-21 at 16:52 +0300, Michael S. Tsirkin wrote: + rcu_read_lock(); + mapping = rcu_dereference(page-mapping); + if (mapping_balloon(mapping)) + ret =

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Peter Zijlstra
On Tue, 2012-08-21 at 16:52 +0300, Michael S. Tsirkin wrote: + rcu_read_lock(); + mapping = rcu_dereference(page-mapping); + if (mapping_balloon(mapping)) + ret = true; + rcu_read_unlock(); This looks suspicious: you

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Peter Zijlstra
On Tue, 2012-08-21 at 09:47 -0300, Rafael Aquini wrote: + mapping = rcu_access_pointer(page-mapping); + if (mapping) + mapping = mapping-assoc_mapping; The comment near rcu_access_pointer() explicitly says: * Return the value of the specified RCU-protected pointer,

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Paul E. McKenney
On Tue, Aug 21, 2012 at 05:20:11PM +0200, Peter Zijlstra wrote: On Tue, 2012-08-21 at 09:47 -0300, Rafael Aquini wrote: + mapping = rcu_access_pointer(page-mapping); + if (mapping) + mapping = mapping-assoc_mapping; The comment near rcu_access_pointer()

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 06:41:42PM +0300, Michael S. Tsirkin wrote: On Tue, Aug 21, 2012 at 05:16:06PM +0200, Peter Zijlstra wrote: On Tue, 2012-08-21 at 16:52 +0300, Michael S. Tsirkin wrote: + rcu_read_lock(); + mapping = rcu_dereference(page-mapping); +

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 04:52:23PM +0300, Michael S. Tsirkin wrote: + * address_space_operations utilized methods for ballooned pages: + * .migratepage- used to perform balloon's page migration (as is) + * .launder_page - used to isolate a page from balloon's page list + *

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 09:24:32AM -0700, Paul E. McKenney wrote: On Tue, Aug 21, 2012 at 05:20:11PM +0200, Peter Zijlstra wrote: On Tue, 2012-08-21 at 09:47 -0300, Rafael Aquini wrote: + mapping = rcu_access_pointer(page-mapping); + if (mapping) + mapping =

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Michael S. Tsirkin
On Tue, Aug 21, 2012 at 02:28:20PM -0300, Rafael Aquini wrote: On Tue, Aug 21, 2012 at 09:24:32AM -0700, Paul E. McKenney wrote: On Tue, Aug 21, 2012 at 05:20:11PM +0200, Peter Zijlstra wrote: On Tue, 2012-08-21 at 09:47 -0300, Rafael Aquini wrote: + mapping =

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Michael S. Tsirkin
On Tue, Aug 21, 2012 at 02:55:03PM -0300, Rafael Aquini wrote: On Tue, Aug 21, 2012 at 04:52:23PM +0300, Michael S. Tsirkin wrote: + * address_space_operations utilized methods for ballooned pages: + * .migratepage- used to perform balloon's page migration (as is) + *

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 10:13:30PM +0300, Michael S. Tsirkin wrote: I believe rcu_dereference_protected() is what I want/need here, since this code is always called for pages which we hold locked (PG_locked bit). It would only help if we locked the page while updating the mapping, as

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Michael S. Tsirkin
On Tue, Aug 21, 2012 at 02:42:52PM -0300, Rafael Aquini wrote: On Tue, Aug 21, 2012 at 06:41:42PM +0300, Michael S. Tsirkin wrote: On Tue, Aug 21, 2012 at 05:16:06PM +0200, Peter Zijlstra wrote: On Tue, 2012-08-21 at 16:52 +0300, Michael S. Tsirkin wrote: + rcu_read_lock();

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Michael S. Tsirkin
On Tue, Aug 21, 2012 at 04:23:58PM -0300, Rafael Aquini wrote: On Tue, Aug 21, 2012 at 10:13:30PM +0300, Michael S. Tsirkin wrote: I believe rcu_dereference_protected() is what I want/need here, since this code is always called for pages which we hold locked (PG_locked bit). It

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 10:16:12PM +0300, Michael S. Tsirkin wrote: On Tue, Aug 21, 2012 at 02:55:03PM -0300, Rafael Aquini wrote: On Tue, Aug 21, 2012 at 04:52:23PM +0300, Michael S. Tsirkin wrote: + * address_space_operations utilized methods for ballooned pages: + * .migratepage

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 10:30:31PM +0300, Michael S. Tsirkin wrote: On Tue, Aug 21, 2012 at 04:23:58PM -0300, Rafael Aquini wrote: On Tue, Aug 21, 2012 at 10:13:30PM +0300, Michael S. Tsirkin wrote: I believe rcu_dereference_protected() is what I want/need here, since this code

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Michael S. Tsirkin
On Tue, Aug 21, 2012 at 04:34:39PM -0300, Rafael Aquini wrote: On Tue, Aug 21, 2012 at 10:16:12PM +0300, Michael S. Tsirkin wrote: On Tue, Aug 21, 2012 at 02:55:03PM -0300, Rafael Aquini wrote: On Tue, Aug 21, 2012 at 04:52:23PM +0300, Michael S. Tsirkin wrote: + *

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Wed, Aug 22, 2012 at 03:07:41AM +0300, Michael S. Tsirkin wrote: On Tue, Aug 21, 2012 at 05:45:56PM -0300, Rafael Aquini wrote: On Tue, Aug 21, 2012 at 10:30:31PM +0300, Michael S. Tsirkin wrote: On Tue, Aug 21, 2012 at 04:23:58PM -0300, Rafael Aquini wrote: On Tue, Aug 21, 2012 at