Re: [FYI] tux3: Core changes

2015-08-18 Thread Rik van Riel
On 07/31/2015 01:27 PM, Daniel Phillips wrote: On Friday, July 31, 2015 8:37:35 AM PDT, Raymond Jennings wrote: Returning ENOSPC when you have free space you can't yet prove is safer than not returning it and risking a data loss when you get hit by a write/commit storm. :) Remember when

Re: [FYI] tux3: Core changes

2015-08-16 Thread OGAWA Hirofumi
Jan Kara j...@suse.cz writes: On Sun 09-08-15 22:42:42, OGAWA Hirofumi wrote: Jan Kara j...@suse.cz writes: I'm not sure about which ENOSPC issue you are speaking BTW. Can you please ellaborate? 1. GUP simulate page fault, and prepare to modify 2. writeback clear dirty, and make PTE

Re: [FYI] tux3: Core changes

2015-08-10 Thread Jan Kara
On Sun 09-08-15 22:42:42, OGAWA Hirofumi wrote: Jan Kara j...@suse.cz writes: I'm not sure about which ENOSPC issue you are speaking BTW. Can you please ellaborate? 1. GUP simulate page fault, and prepare to modify 2. writeback clear dirty, and make PTE read-only 3. snapshot/reflink

Re: [FYI] tux3: Core changes

2015-08-09 Thread OGAWA Hirofumi
Jan Kara j...@suse.cz writes: I'm not sure about which ENOSPC issue you are speaking BTW. Can you please ellaborate? 1. GUP simulate page fault, and prepare to modify 2. writeback clear dirty, and make PTE read-only 3. snapshot/reflink make block cow 4. driver called GUP modifies page, and

Re: [FYI] tux3: Core changes

2015-08-03 Thread Jan Kara
On Fri 31-07-15 17:16:45, Daniel Phillips wrote: On Friday, July 31, 2015 5:00:43 PM PDT, Daniel Phillips wrote: Note: Hirofumi's email is clear, logical and speaks to the question. This branch of the thread is largely pointless, though it essentially says the same thing in non-technical

Re: [FYI] tux3: Core changes

2015-08-01 Thread Elifarley Callado Coelho Cruz
My gosh This is driving me crazy. Please let's make it crystal clear, in technical and precise terms, devoid of any ad hominem attacks or the like, what is preventing Tux3 from being merged. Maybe a list of issues in Github, so that each issue can be scrutinized more easily. Like this one:

Re: [FYI] tux3: Core changes

2015-07-31 Thread Daniel Phillips
On Friday, July 31, 2015 5:00:43 PM PDT, Daniel Phillips wrote: Note: Hirofumi's email is clear, logical and speaks to the question. This branch of the thread is largely pointless, though it essentially says the same thing in non-technical terms. Perhaps your next response should be to Hirofumi,

Re: [FYI] tux3: Core changes

2015-07-31 Thread Raymond Jennings
Returning ENOSPC when you have free space you can't yet prove is safer than not returning it and risking a data loss when you get hit by a write/commit storm. :) On Thu, Jul 30, 2015 at 9:44 PM, OGAWA Hirofumi hirof...@mail.parknet.co.jp wrote: Jan Kara j...@suse.cz writes: Yes, if

Re: [FYI] tux3: Core changes

2015-07-31 Thread Daniel Phillips
On Friday, July 31, 2015 11:29:51 AM PDT, David Lang wrote: If you define this as loosing our mojo, then yes we have. A pity. There remains so much to do that simply will not get done in the absence of mojo. Regards, Daniel ___ Tux3 mailing list

Re: [FYI] tux3: Core changes

2015-07-31 Thread David Lang
On Fri, 31 Jul 2015, Daniel Phillips wrote: Subject: Re: [FYI] tux3: Core changes On Friday, July 31, 2015 8:37:35 AM PDT, Raymond Jennings wrote: Returning ENOSPC when you have free space you can't yet prove is safer than not returning it and risking a data loss when you get hit by a write

Re: [FYI] tux3: Core changes

2015-07-31 Thread Daniel Phillips
On Friday, July 31, 2015 11:29:51 AM PDT, David Lang wrote: We, the Linux Community have less tolerance for losing people's data and preventing them from operating than we used to when it was all tinkerer's personal data and secondary systems. So rather than pushing optimizations out to

Re: [FYI] tux3: Core changes

2015-07-31 Thread Daniel Phillips
On Friday, July 31, 2015 3:27:12 PM PDT, David Lang wrote: On Fri, 31 Jul 2015, Daniel Phillips wrote: On Friday, July 31, 2015 11:29:51 AM PDT, David Lang wrote: ... you weren't asking about any particular feature of Tux, you were asking if we were still willing to push out stuff that

Re: [FYI] tux3: Core changes

2015-07-31 Thread Daniel Phillips
On Friday, July 31, 2015 11:29:51 AM PDT, David Lang wrote: We, the Linux Community have less tolerance for losing people's data and preventing them from operating than we used to when it was all tinkerer's personal data and secondary systems. So rather than pushing optimizations out to

Re: [FYI] tux3: Core changes

2015-07-31 Thread David Lang
On Fri, 31 Jul 2015, Daniel Phillips wrote: On Friday, July 31, 2015 11:29:51 AM PDT, David Lang wrote: We, the Linux Community have less tolerance for losing people's data and preventing them from operating than we used to when it was all tinkerer's personal data and secondary systems. So

Re: [FYI] tux3: Core changes

2015-07-30 Thread OGAWA Hirofumi
Jan Kara j...@suse.cz writes: Yes, if userspace truncates the file, the situation we end up with is basically the same. However for truncate to happen some malicious process has to come and truncate the file - a failure scenario that is acceptable for most use cases since it doesn't

Re: [FYI] tux3: Core changes

2015-07-09 Thread Jan Kara
On Sun 05-07-15 21:54:45, OGAWA Hirofumi wrote: Jan Kara j...@suse.cz writes: I'm not sure I'm understanding your pseudocode logic correctly though. This logic doesn't seems to be a page forking specific issue. And this pseudocode logic seems to be missing the locking and revalidate of

Re: [FYI] tux3: Core changes

2015-07-05 Thread OGAWA Hirofumi
Jan Kara j...@suse.cz writes: I'm not sure I'm understanding your pseudocode logic correctly though. This logic doesn't seems to be a page forking specific issue. And this pseudocode logic seems to be missing the locking and revalidate of page. If you can show more details, it would be

Re: [FYI] tux3: Core changes

2015-06-23 Thread Jan Kara
On Mon 22-06-15 00:36:00, OGAWA Hirofumi wrote: Jan Kara j...@suse.cz writes: So there are a few things to have in mind: 1) There is nothing like a writeable page. Page is always writeable (at least on x86 architecture). When a page is mapped into some virtual address space (or more of

Re: [FYI] tux3: Core changes

2015-06-21 Thread OGAWA Hirofumi
Jan Kara j...@suse.cz writes: Hi, So there are a few things to have in mind: 1) There is nothing like a writeable page. Page is always writeable (at least on x86 architecture). When a page is mapped into some virtual address space (or more of them), this *mapping* can be either writeable or

Re: [FYI] tux3: Core changes

2015-05-27 Thread Pavel Machek
On Wed 2015-05-27 11:09:25, Daniel Phillips wrote: On Wednesday, May 27, 2015 12:41:37 AM PDT, Pavel Machek wrote: On Fri 2015-05-15 02:38:33, Daniel Phillips wrote: On 05/14/2015 08:06 PM, Rik van Riel wrote: ... Umm. Why do you think it is only issue for executable files? I meant: files

Re: [FYI] tux3: Core changes

2015-05-27 Thread Daniel Phillips
On 05/27/2015 02:37 PM, Pavel Machek wrote: On Wed 2015-05-27 11:09:25, Daniel Phillips wrote: On Wednesday, May 27, 2015 12:41:37 AM PDT, Pavel Machek wrote: On Fri 2015-05-15 02:38:33, Daniel Phillips wrote: On 05/14/2015 08:06 PM, Rik van Riel wrote: ... Umm. Why do you think it is only

Re: [FYI] tux3: Core changes

2015-05-26 Thread Jan Kara
On Tue 26-05-15 01:08:56, Daniel Phillips wrote: On Tuesday, May 26, 2015 12:09:10 AM PDT, Jan Kara wrote: E.g. video drivers (or infiniband or direct IO for that matter) which have buffers in user memory (may be mmapped file), grab references to pages and hand out PFNs of those pages to the

Re: [FYI] tux3: Core changes

2015-05-26 Thread Jan Kara
On Mon 25-05-15 23:11:11, Daniel Phillips wrote: On Monday, May 25, 2015 11:04:39 PM PDT, David Lang wrote: if the page gets modified again, will that cause any issues? what if the page gets modified before the copy gets written out, so that there are two dirty copies of the page in the

Re: [FYI] tux3: Core changes

2015-05-26 Thread Sergey Senozhatsky
On (05/26/15 01:08), Daniel Phillips wrote: On Tuesday, May 26, 2015 12:09:10 AM PDT, Jan Kara wrote: E.g. video drivers (or infiniband or direct IO for that matter) which have buffers in user memory (may be mmapped file), grab references to pages and hand out PFNs of those pages to the

Re: [FYI] tux3: Core changes

2015-05-26 Thread Daniel Phillips
On Tuesday, May 26, 2015 3:13:02 AM PDT, Pavel Machek wrote: On Tue 2015-05-26 01:09:59, Daniel Phillips wrote: On Monday, May 25, 2015 11:13:46 PM PDT, David Lang wrote: I'm assuming that Rik is talking about whatever has the reference to the page via one of the methods that he talked about.

Re: [FYI] tux3: Core changes

2015-05-26 Thread Daniel Phillips
On 05/26/2015 02:00 AM, Jan Kara wrote: On Tue 26-05-15 01:08:56, Daniel Phillips wrote: On Tuesday, May 26, 2015 12:09:10 AM PDT, Jan Kara wrote: E.g. video drivers (or infiniband or direct IO for that matter) which have buffers in user memory (may be mmapped file), grab references to pages

Re: [FYI] tux3: Core changes

2015-05-26 Thread Daniel Phillips
Hi Sergey, On 05/26/2015 03:22 AM, Sergey Senozhatsky wrote: Hello, is it possible to page-fork-bomb the system by some 'malicious' app? Not in any new way. A page fork can happen either in the front end, where it has to wait for memory like any other normal memory user, or in the backend,

Re: [FYI] tux3: Core changes

2015-05-26 Thread Rik van Riel
On 05/26/2015 04:22 PM, Daniel Phillips wrote: On 05/26/2015 02:00 AM, Jan Kara wrote: So my opinion is: Don't fork the page if page_count is elevated. You can just wait for the IO if you need stable pages in that case. It's slow but it's safe and it should be pretty rare. Is there any

Re: [FYI] tux3: Core changes

2015-05-20 Thread Daniel Phillips
On 05/20/2015 07:44 AM, Jan Kara wrote: On Tue 19-05-15 13:33:31, David Lang wrote: On Tue, 19 May 2015, Daniel Phillips wrote: I understand that Tux3 may avoid these issues due to some other mechanisms it internally has but if page forking should get into mm subsystem, the above must

Re: [FYI] tux3: Core changes

2015-05-20 Thread David Lang
On Wed, 20 May 2015, Daniel Phillips wrote: On 05/20/2015 03:51 PM, Daniel Phillips wrote: On 05/20/2015 12:53 PM, Rik van Riel wrote: How does tux3 prevent a user of find_get_page() from reading from or writing into the pre-COW page, instead of the current page? Careful control of the

Re: [FYI] tux3: Core changes

2015-05-19 Thread David Lang
On Tue, 19 May 2015, Daniel Phillips wrote: I understand that Tux3 may avoid these issues due to some other mechanisms it internally has but if page forking should get into mm subsystem, the above must work. It does work, and by example, it does not need a lot of code to make it work, but the

Re: [FYI] tux3: Core changes

2015-05-19 Thread Jan Kara
On Thu 14-05-15 01:26:23, Daniel Phillips wrote: Hi Rik, Our linux-tux3 tree currently currently carries this 652 line diff against core, to make Tux3 work. This is mainly by Hirofumi, except the fs-writeback.c hook, which is by me. The main part you may be interested in is rmap.c, which

Re: [FYI] tux3: Core changes

2015-05-18 Thread Mel Gorman
On Sat, May 16, 2015 at 03:38:04PM -0700, David Lang wrote: On Fri, 15 May 2015, Mel Gorman wrote: On Fri, May 15, 2015 at 02:54:48AM -0700, Daniel Phillips wrote: On 05/15/2015 01:09 AM, Mel Gorman wrote: On Thu, May 14, 2015 at 11:06:22PM -0400, Rik van Riel wrote: On 05/14/2015 08:06

Re: [FYI] tux3: Core changes

2015-05-18 Thread Boaz Harrosh
On 05/18/2015 05:20 AM, Rik van Riel wrote: On 05/17/2015 09:26 AM, Boaz Harrosh wrote: On 05/14/2015 03:59 PM, Rik van Riel wrote: On 05/14/2015 04:26 AM, Daniel Phillips wrote: Hi Rik, The issue is that things like ptrace, AIO, infiniband RDMA, and other direct memory access subsystems

Re: [FYI] tux3: Core changes

2015-05-18 Thread Boaz Harrosh
On 05/14/2015 03:59 PM, Rik van Riel wrote: On 05/14/2015 04:26 AM, Daniel Phillips wrote: Hi Rik, The issue is that things like ptrace, AIO, infiniband RDMA, and other direct memory access subsystems can take a reference to page A, which Tux3 clones into a new page B when the process

Re: [FYI] tux3: Core changes

2015-05-18 Thread Daniel Phillips
On 05/17/2015 07:20 PM, Rik van Riel wrote: On 05/17/2015 09:26 AM, Boaz Harrosh wrote: On 05/14/2015 03:59 PM, Rik van Riel wrote: The issue is that things like ptrace, AIO, infiniband RDMA, and other direct memory access subsystems can take a reference to page A, which Tux3 clones into a

Re: [FYI] tux3: Core changes

2015-05-17 Thread Rik van Riel
On 05/17/2015 09:26 AM, Boaz Harrosh wrote: On 05/14/2015 03:59 PM, Rik van Riel wrote: On 05/14/2015 04:26 AM, Daniel Phillips wrote: Hi Rik, The issue is that things like ptrace, AIO, infiniband RDMA, and other direct memory access subsystems can take a reference to page A, which Tux3

Re: [FYI] tux3: Core changes

2015-05-16 Thread David Lang
On Fri, 15 May 2015, Mel Gorman wrote: On Fri, May 15, 2015 at 02:54:48AM -0700, Daniel Phillips wrote: On 05/15/2015 01:09 AM, Mel Gorman wrote: On Thu, May 14, 2015 at 11:06:22PM -0400, Rik van Riel wrote: On 05/14/2015 08:06 PM, Daniel Phillips wrote: The issue is that things like

Re: [FYI] tux3: Core changes

2015-05-15 Thread Mel Gorman
On Thu, May 14, 2015 at 05:06:39PM -0700, Daniel Phillips wrote: Hi Rik, Added Mel, Andrea and Peterz to CC as interested parties. There are probably others, please just jump in. On 05/14/2015 05:59 AM, Rik van Riel wrote: On 05/14/2015 04:26 AM, Daniel Phillips wrote: Hi Rik, Our

Re: [FYI] tux3: Core changes

2015-05-15 Thread Daniel Phillips
On 05/15/2015 01:09 AM, Mel Gorman wrote: On Thu, May 14, 2015 at 11:06:22PM -0400, Rik van Riel wrote: On 05/14/2015 08:06 PM, Daniel Phillips wrote: The issue is that things like ptrace, AIO, infiniband RDMA, and other direct memory access subsystems can take a reference to page A, which

Re: [FYI] tux3: Core changes

2015-05-15 Thread Mel Gorman
On Fri, May 15, 2015 at 02:54:48AM -0700, Daniel Phillips wrote: On 05/15/2015 01:09 AM, Mel Gorman wrote: On Thu, May 14, 2015 at 11:06:22PM -0400, Rik van Riel wrote: On 05/14/2015 08:06 PM, Daniel Phillips wrote: The issue is that things like ptrace, AIO, infiniband RDMA, and other

Re: [FYI] tux3: Core changes

2015-05-14 Thread Rik van Riel
On 05/14/2015 04:26 AM, Daniel Phillips wrote: Hi Rik, Our linux-tux3 tree currently currently carries this 652 line diff against core, to make Tux3 work. This is mainly by Hirofumi, except the fs-writeback.c hook, which is by me. The main part you may be interested in is rmap.c, which