Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-24 Thread Michael S. Tsirkin
On Thu, Aug 23, 2012 at 11:12:17PM -0400, Rik van Riel wrote: > On 08/23/2012 07:36 PM, Michael S. Tsirkin wrote: > > >---> > > > >virtio-balloon: replace page->lru list with page->private. > > > >The point is to free up page->lru for use by compaction. > >Warning: completely untested, will provid

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rik van Riel
On 08/23/2012 07:36 PM, Michael S. Tsirkin wrote: ---> virtio-balloon: replace page->lru list with page->private. The point is to free up page->lru for use by compaction. Warning: completely untested, will provide tested version if we agree on this direction. A singly linked list is not goin

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 09:38:48PM -0300, Rafael Aquini wrote: > On Thu, Aug 23, 2012 at 09:33:53PM -0300, Rafael Aquini wrote: > > On Fri, Aug 24, 2012 at 02:36:16AM +0300, Michael S. Tsirkin wrote: > > > I would wake it each time after adding a page, then it > > > can stop waiting when it leaks e

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 09:33:53PM -0300, Rafael Aquini wrote: > On Fri, Aug 24, 2012 at 02:36:16AM +0300, Michael S. Tsirkin wrote: > > I would wake it each time after adding a page, then it > > can stop waiting when it leaks enough. > > But again, it's cleaner to just keep tracking all > > pages,

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Fri, Aug 24, 2012 at 02:36:16AM +0300, Michael S. Tsirkin wrote: > I would wake it each time after adding a page, then it > can stop waiting when it leaks enough. > But again, it's cleaner to just keep tracking all > pages, let mm hang on to them by keeping a reference. > Here is a rough idea o

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Fri, Aug 24, 2012 at 02:36:16AM +0300, Michael S. Tsirkin wrote: > On Thu, Aug 23, 2012 at 02:28:45PM -0300, Rafael Aquini wrote: > > On Thu, Aug 23, 2012 at 07:25:05PM +0300, Michael S. Tsirkin wrote: > > > On Thu, Aug 23, 2012 at 04:53:28PM +0300, Michael S. Tsirkin wrote: > > > > Basically it

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Michael S. Tsirkin
On Thu, Aug 23, 2012 at 02:28:45PM -0300, Rafael Aquini wrote: > On Thu, Aug 23, 2012 at 07:25:05PM +0300, Michael S. Tsirkin wrote: > > On Thu, Aug 23, 2012 at 04:53:28PM +0300, Michael S. Tsirkin wrote: > > > Basically it was very simple: we assumed page->lru was never > > > touched for an alloca

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rik van Riel
On 08/23/2012 01:28 PM, Rafael Aquini wrote: What about keep a conter at virtio_balloon structure on how much pages are isolated from balloon's list and check it at leak time? if the counter gets > 0 than we can safely put leak_balloon() to wait until balloon page list gets completely refilled.

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 07:25:05PM +0300, Michael S. Tsirkin wrote: > On Thu, Aug 23, 2012 at 04:53:28PM +0300, Michael S. Tsirkin wrote: > > Basically it was very simple: we assumed page->lru was never > > touched for an allocated page, so it's safe to use it for > > internal book-keeping by the d

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Michael S. Tsirkin
On Thu, Aug 23, 2012 at 04:53:28PM +0300, Michael S. Tsirkin wrote: > Basically it was very simple: we assumed page->lru was never > touched for an allocated page, so it's safe to use it for > internal book-keeping by the driver. > > Now, this is not the case anymore, you add some logic in mm/ tha

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Michael S. Tsirkin
On Thu, Aug 23, 2012 at 01:06:48PM -0300, Rafael Aquini wrote: > On Thu, Aug 23, 2012 at 12:03:15PM -0400, Rik van Riel wrote: > > > > > >Not "longer" - apparently forever unless user resend the leak command. > > >It's wrong - it should > > >1. not tell host if nothing was done > > >2. after migrat

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 12:03:15PM -0400, Rik van Riel wrote: > > > >Not "longer" - apparently forever unless user resend the leak command. > >It's wrong - it should > >1. not tell host if nothing was done > >2. after migration finished leak and tell host > > Agreed. If the balloon is told to lea

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rik van Riel
On 08/23/2012 11:54 AM, Michael S. Tsirkin wrote: On Thu, Aug 23, 2012 at 12:21:29PM -0300, Rafael Aquini wrote: On Thu, Aug 23, 2012 at 04:53:29PM +0300, Michael S. Tsirkin wrote: On Thu, Aug 23, 2012 at 10:06:07AM -0300, Rafael Aquini wrote: On Thu, Aug 23, 2012 at 03:34:32PM +0300, Michael

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Michael S. Tsirkin
On Thu, Aug 23, 2012 at 12:21:29PM -0300, Rafael Aquini wrote: > On Thu, Aug 23, 2012 at 04:53:29PM +0300, Michael S. Tsirkin wrote: > > On Thu, Aug 23, 2012 at 10:06:07AM -0300, Rafael Aquini wrote: > > > On Thu, Aug 23, 2012 at 03:34:32PM +0300, Michael S. Tsirkin wrote: > > > > > So, nothing has

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 04:53:29PM +0300, Michael S. Tsirkin wrote: > On Thu, Aug 23, 2012 at 10:06:07AM -0300, Rafael Aquini wrote: > > On Thu, Aug 23, 2012 at 03:34:32PM +0300, Michael S. Tsirkin wrote: > > > > So, nothing has changed here. > > > > > > Yes, your patch does change things: > > > l

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Michael S. Tsirkin
On Thu, Aug 23, 2012 at 10:06:07AM -0300, Rafael Aquini wrote: > On Thu, Aug 23, 2012 at 03:34:32PM +0300, Michael S. Tsirkin wrote: > > > So, nothing has changed here. > > > > Yes, your patch does change things: > > leak_balloon now might return without freeing any pages. > > In that case we will

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 03:34:32PM +0300, Michael S. Tsirkin wrote: > > So, nothing has changed here. > > Yes, your patch does change things: > leak_balloon now might return without freeing any pages. > In that case we will not be making any progress, and just > spin, pinning CPU. That's a transi

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Michael S. Tsirkin
On Thu, Aug 23, 2012 at 09:13:39AM -0300, Rafael Aquini wrote: > On Thu, Aug 23, 2012 at 01:01:07PM +0300, Michael S. Tsirkin wrote: > > > So, when remove_common() calls leak_balloon() looping on > > > vb->num_pages, that won't become a tight loop. > > > The scheme was apparently working before th

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 01:01:07PM +0300, Michael S. Tsirkin wrote: > > So, when remove_common() calls leak_balloon() looping on > > vb->num_pages, that won't become a tight loop. > > The scheme was apparently working before this series, and it will remain > > working > > after it. > > It seems

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Michael S. Tsirkin
On Wed, Aug 22, 2012 at 11:19:04PM -0300, Rafael Aquini wrote: > On Wed, Aug 22, 2012 at 12:33:17PM +0300, Michael S. Tsirkin wrote: > > Hmm, so this will busy wait which is unelegant. > > We need some event IMO. > > No, it does not busy wait. leak_balloon() is mutual exclusive with migration > st

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-22 Thread Rafael Aquini
On Wed, Aug 22, 2012 at 12:33:17PM +0300, Michael S. Tsirkin wrote: > Hmm, so this will busy wait which is unelegant. > We need some event IMO. No, it does not busy wait. leak_balloon() is mutual exclusive with migration steps, so for the case we have one racing against the other, we really want l

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-22 Thread Michael S. Tsirkin
On Tue, Aug 21, 2012 at 10:19:31PM -0300, Rafael Aquini wrote: > On Wed, Aug 22, 2012 at 03:07:41AM +0300, Michael S. Tsirkin wrote: > > On Tue, Aug 21, 2012 at 05:45:56PM -0300, Rafael Aquini wrote: > > > On Tue, Aug 21, 2012 at 10:30:31PM +0300, Michael S. Tsirkin wrote: > > > > On Tue, Aug 21, 2

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Wed, Aug 22, 2012 at 03:07:41AM +0300, Michael S. Tsirkin wrote: > On Tue, Aug 21, 2012 at 05:45:56PM -0300, Rafael Aquini wrote: > > On Tue, Aug 21, 2012 at 10:30:31PM +0300, Michael S. Tsirkin wrote: > > > On Tue, Aug 21, 2012 at 04:23:58PM -0300, Rafael Aquini wrote: > > > > On Tue, Aug 21, 2

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Michael S. Tsirkin
On Tue, Aug 21, 2012 at 05:45:56PM -0300, Rafael Aquini wrote: > On Tue, Aug 21, 2012 at 10:30:31PM +0300, Michael S. Tsirkin wrote: > > On Tue, Aug 21, 2012 at 04:23:58PM -0300, Rafael Aquini wrote: > > > On Tue, Aug 21, 2012 at 10:13:30PM +0300, Michael S. Tsirkin wrote: > > > > > > > > > > I be

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Michael S. Tsirkin
On Tue, Aug 21, 2012 at 04:34:39PM -0300, Rafael Aquini wrote: > On Tue, Aug 21, 2012 at 10:16:12PM +0300, Michael S. Tsirkin wrote: > > On Tue, Aug 21, 2012 at 02:55:03PM -0300, Rafael Aquini wrote: > > > On Tue, Aug 21, 2012 at 04:52:23PM +0300, Michael S. Tsirkin wrote: > > > > > + * address_spa

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 10:30:31PM +0300, Michael S. Tsirkin wrote: > On Tue, Aug 21, 2012 at 04:23:58PM -0300, Rafael Aquini wrote: > > On Tue, Aug 21, 2012 at 10:13:30PM +0300, Michael S. Tsirkin wrote: > > > > > > > > I believe rcu_dereference_protected() is what I want/need here, since > > >

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 10:16:12PM +0300, Michael S. Tsirkin wrote: > On Tue, Aug 21, 2012 at 02:55:03PM -0300, Rafael Aquini wrote: > > On Tue, Aug 21, 2012 at 04:52:23PM +0300, Michael S. Tsirkin wrote: > > > > + * address_space_operations utilized methods for ballooned pages: > > > > + * .migr

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Michael S. Tsirkin
On Tue, Aug 21, 2012 at 04:23:58PM -0300, Rafael Aquini wrote: > On Tue, Aug 21, 2012 at 10:13:30PM +0300, Michael S. Tsirkin wrote: > > > > > > I believe rcu_dereference_protected() is what I want/need here, since > > > this code > > > is always called for pages which we hold locked (PG_locked b

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Michael S. Tsirkin
On Tue, Aug 21, 2012 at 02:42:52PM -0300, Rafael Aquini wrote: > On Tue, Aug 21, 2012 at 06:41:42PM +0300, Michael S. Tsirkin wrote: > > On Tue, Aug 21, 2012 at 05:16:06PM +0200, Peter Zijlstra wrote: > > > On Tue, 2012-08-21 at 16:52 +0300, Michael S. Tsirkin wrote: > > > > > + rcu_rea

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 10:13:30PM +0300, Michael S. Tsirkin wrote: > > > > I believe rcu_dereference_protected() is what I want/need here, since this > > code > > is always called for pages which we hold locked (PG_locked bit). > > It would only help if we locked the page while updating the map

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Michael S. Tsirkin
On Tue, Aug 21, 2012 at 02:55:03PM -0300, Rafael Aquini wrote: > On Tue, Aug 21, 2012 at 04:52:23PM +0300, Michael S. Tsirkin wrote: > > > + * address_space_operations utilized methods for ballooned pages: > > > + * .migratepage- used to perform balloon's page migration (as is) > > > + * .l

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Michael S. Tsirkin
On Tue, Aug 21, 2012 at 02:28:20PM -0300, Rafael Aquini wrote: > On Tue, Aug 21, 2012 at 09:24:32AM -0700, Paul E. McKenney wrote: > > On Tue, Aug 21, 2012 at 05:20:11PM +0200, Peter Zijlstra wrote: > > > On Tue, 2012-08-21 at 09:47 -0300, Rafael Aquini wrote: > > > > + mapping = rcu_access_p

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 09:24:32AM -0700, Paul E. McKenney wrote: > On Tue, Aug 21, 2012 at 05:20:11PM +0200, Peter Zijlstra wrote: > > On Tue, 2012-08-21 at 09:47 -0300, Rafael Aquini wrote: > > > + mapping = rcu_access_pointer(page->mapping); > > > + if (mapping) > > > +

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 06:41:42PM +0300, Michael S. Tsirkin wrote: > On Tue, Aug 21, 2012 at 05:16:06PM +0200, Peter Zijlstra wrote: > > On Tue, 2012-08-21 at 16:52 +0300, Michael S. Tsirkin wrote: > > > > + rcu_read_lock(); > > > > + mapping = rcu_dereference(page->mapping

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 04:52:23PM +0300, Michael S. Tsirkin wrote: > > + * address_space_operations utilized methods for ballooned pages: > > + * .migratepage- used to perform balloon's page migration (as is) > > + * .launder_page - used to isolate a page from balloon's page list > > + *

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Paul E. McKenney
On Tue, Aug 21, 2012 at 05:20:11PM +0200, Peter Zijlstra wrote: > On Tue, 2012-08-21 at 09:47 -0300, Rafael Aquini wrote: > > + mapping = rcu_access_pointer(page->mapping); > > + if (mapping) > > + mapping = mapping->assoc_mapping; > > The comment near rcu_access_pointer

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Peter Zijlstra
On Tue, 2012-08-21 at 09:47 -0300, Rafael Aquini wrote: > + mapping = rcu_access_pointer(page->mapping); > + if (mapping) > + mapping = mapping->assoc_mapping; The comment near rcu_access_pointer() explicitly says: * Return the value of the specified RCU-protected poin

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Peter Zijlstra
On Tue, 2012-08-21 at 16:52 +0300, Michael S. Tsirkin wrote: > > + rcu_read_lock(); > > + mapping = rcu_dereference(page->mapping); > > + if (mapping_balloon(mapping)) > > + ret = true; > > + rcu_read_unlock(); > > This looks susp

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Michael S. Tsirkin
On Tue, Aug 21, 2012 at 05:16:06PM +0200, Peter Zijlstra wrote: > On Tue, 2012-08-21 at 16:52 +0300, Michael S. Tsirkin wrote: > > > + rcu_read_lock(); > > > + mapping = rcu_dereference(page->mapping); > > > + if (mapping_balloon(mapping)) > > > +

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Michael S. Tsirkin
On Tue, Aug 21, 2012 at 04:52:23PM +0300, Michael S. Tsirkin wrote: > On Tue, Aug 21, 2012 at 09:47:44AM -0300, Rafael Aquini wrote: > > Memory fragmentation introduced by ballooning might reduce significantly > > the number of 2MB contiguous memory blocks that can be used within a guest, > > thus

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Michael S. Tsirkin
On Tue, Aug 21, 2012 at 09:47:44AM -0300, Rafael Aquini wrote: > Memory fragmentation introduced by ballooning might reduce significantly > the number of 2MB contiguous memory blocks that can be used within a guest, > thus imposing performance penalties associated with the reduced number of > trans

[PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
Memory fragmentation introduced by ballooning might reduce significantly the number of 2MB contiguous memory blocks that can be used within a guest, thus imposing performance penalties associated with the reduced number of transparent huge pages that could be used by the guest workload. This patch