Re: RFC: Case-insensitive support for XFS

2007-10-08 Thread Barry Naujok
On Mon, 08 Oct 2007 15:44:48 +1000, Nicholas Miell [EMAIL PROTECTED]  
wrote:



On Mon, 2007-10-08 at 15:07 +1000, Barry Naujok wrote:

On Sat, 06 Oct 2007 04:52:18 +1000, Nicholas Miell [EMAIL PROTECTED]
wrote:

 On Fri, 2007-10-05 at 16:44 +0100, Christoph Hellwig wrote:
 [Adding -fsdevel because some of the things touched here might be of
  broader interest and Urban because his name is on nls_utf8.c]

 On Fri, Oct 05, 2007 at 11:57:54AM +1000, Barry Naujok wrote:
 
  On it's own, linux only provides case conversion for old-style
  character sets - 8 bit sequences only. A lot of distos are
  now defaulting to UTF-8 and Linux NLS stuff does not support
  case conversion for any unicode sets.

 The lack of case tables in nls_utf8.c defintively seems odd to me.
 Urban, is there a reason for that?  The only thing that comes to
 mind is that these tables might be quite large.


 Case conversion in Unicode is locale dependent. The legacy 8-bit
 character encodings don't code for enough characters to run into the
 ambiguities, so they can get away with fixed case conversion tables.
 Unicode can't.

Based on http://www.unicode.org/reports/tr21/tr21-5.html and
http://www.unicode.org/Public/UNIDATA/CaseFolding.txt

Doing case comparison using that table should cater for most
circumstances except a few exeptions. It should be enough
to satisfy a locale independant case-insensitive filesystem
(ie. the C + F case folding option).

Is normalization required after case-folding? What I read
implies it is not necessary for this purpose (and would
slow things down and bloat the code more).

Now I suppose, it's just a question of a fixed table in the
kernel driver (HFS+ style), or data stored in a special
inode on-disk (NTFS style, shared refcounted in memory
when the same). With the on-disk, the table can be generated
 from mkfs.xfs.


You also have to decide whether to screw over people who speak Turkic
languages and expect an 'I' to 'ı' mapping or everybody else who expect
an 'I' to 'i' mapping.


Is there some way in the kernel, that I'm unaware of, in knowing what
the user's current language and/or codepage locale is set to?

The only thing I've found is the isocharset option that the other
filesystems use or the default_nls_table() if one isn't specified.
The default one seems to be a CONFIG option.


Although, if you're content in ignoring the kernel's native NLS case
mapping tables (which expect a locale-independent 1-to-1 mapping), you
could just uppercase everything and map both 'i' and 'ı' to 'I'.

Then you have to decide whether things like 'ê' map to 'E' or 'Ê', which
is also locale dependent.


Looking at case-folding, it would be generating lower case equivalent
characters, nls-charset2lower.

Barry.
-
To unsubscribe from this list: send the line unsubscribe linux-fsdevel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[patch] fs: restore nobh

2007-10-08 Thread Nick Piggin
Hi,

This is overdue, sorry. Got a little complicated, and I've been away from
my filesystem test setup so I didn't want ot send it (lucky, coz I found
a bug after more substantial testing).

Anyway, RFC?

---
Implement nobh in new aops. This is a bit tricky.
FWIW, nobh_truncate is now implemented in a way that does not
create blocks in sparse regions, which is a silly thing for it
to have been doing (isn't it?)

ext2 survives fsx and fsstress. jfs is converted as well... ext3
should be easy to do (but not done yet).

Index: linux-2.6/fs/buffer.c
===
--- linux-2.6.orig/fs/buffer.c  2007-10-08 14:09:35.0 +1000
+++ linux-2.6/fs/buffer.c   2007-10-08 16:45:28.0 +1000
@@ -2378,7 +2378,7 @@
 }
 
 /*
- * nobh_prepare_write()'s prereads are special: the buffer_heads are freed
+ * nobh_write_begin()'s prereads are special: the buffer_heads are freed
  * immediately, while under the page lock.  So it needs a special end_io
  * handler which does not touch the bh after unlocking it.
  */
@@ -2388,16 +2388,45 @@
 }
 
 /*
+ * Attach the singly-linked list of buffers created by nobh_write_begin, to
+ * the page (converting it to circular linked list and taking care of page
+ * dirty races).
+ */
+static void attach_nobh_buffers(struct page *page, struct buffer_head *head)
+{
+   struct buffer_head *bh;
+
+   BUG_ON(!PageLocked(page));
+
+   spin_lock(page-mapping-private_lock);
+   bh = head;
+   do {
+   if (PageDirty(page))
+   set_buffer_dirty(bh);
+   if (!bh-b_this_page)
+   bh-b_this_page = head;
+   bh = bh-b_this_page;
+   } while (bh != head);
+   attach_page_buffers(page, head);
+   spin_unlock(page-mapping-private_lock);
+}
+
+/*
  * On entry, the page is fully not uptodate.
  * On exit the page is fully uptodate in the areas outside (from,to)
  */
-int nobh_prepare_write(struct page *page, unsigned from, unsigned to,
+int nobh_write_begin(struct file *file, struct address_space *mapping,
+   loff_t pos, unsigned len, unsigned flags,
+   struct page **pagep, void **fsdata,
get_block_t *get_block)
 {
-   struct inode *inode = page-mapping-host;
+   struct inode *inode = mapping-host;
const unsigned blkbits = inode-i_blkbits;
const unsigned blocksize = 1  blkbits;
struct buffer_head *head, *bh;
+   struct page *page;
+   pgoff_t index;
+   unsigned from, to;
unsigned block_in_page;
unsigned block_start, block_end;
sector_t block_in_file;
@@ -2406,8 +2435,22 @@
int ret = 0;
int is_mapped_to_disk = 1;
 
-   if (page_has_buffers(page))
-   return block_prepare_write(page, from, to, get_block);
+   index = pos  PAGE_CACHE_SHIFT;
+   from = pos  (PAGE_CACHE_SIZE - 1);
+   to = from + len;
+
+   page = __grab_cache_page(mapping, index);
+   if (!page)
+   return -ENOMEM;
+   *pagep = page;
+   *fsdata = NULL;
+
+   if (page_has_buffers(page)) {
+   unlock_page(page);
+   page_cache_release(page);
+   *pagep = NULL;
+   return block_write_begin(file, mapping, pos, len, flags, pagep, 
fsdata, get_block);
+   }
 
if (PageMappedToDisk(page))
return 0;
@@ -2422,8 +2465,10 @@
 * than the circular one we're used to.
 */
head = alloc_page_buffers(page, blocksize, 0);
-   if (!head)
-   return -ENOMEM;
+   if (!head) {
+   ret = -ENOMEM;
+   goto out_release;
+   }
 
block_in_file = (sector_t)page-index  (PAGE_CACHE_SHIFT - blkbits);
 
@@ -2492,15 +2537,12 @@
if (is_mapped_to_disk)
SetPageMappedToDisk(page);
 
-   do {
-   bh = head;
-   head = head-b_this_page;
-   free_buffer_head(bh);
-   } while (head);
+   *fsdata = head; /* to be released by nobh_write_end */
 
return 0;
 
 failed:
+   BUG_ON(!ret);
/*
 * Error recovery is a bit difficult. We need to zero out blocks that
 * were newly allocated, and dirty them to ensure they get written out.
@@ -2508,64 +2550,56 @@
 * the handling of potential IO errors during writeout would be hard
 * (could try doing synchronous writeout, but what if that fails too?)
 */
-   spin_lock(page-mapping-private_lock);
-   bh = head;
-   block_start = 0;
-   do {
-   if (PageUptodate(page))
-   set_buffer_uptodate(bh);
-   if (PageDirty(page))
-   set_buffer_dirty(bh);
+   attach_nobh_buffers(page, head);
+   page_zero_new_buffers(page, from, to);
 
-   block_end = block_start+blocksize;
-   if (block_end 

Re: RFC: Case-insensitive support for XFS

2007-10-08 Thread Barry Naujok
On Mon, 08 Oct 2007 15:44:48 +1000, Nicholas Miell [EMAIL PROTECTED]  
wrote:



You also have to decide whether to screw over people who speak Turkic
languages and expect an 'I' to 'ı' mapping or everybody else who expect
an 'I' to 'i' mapping.


I suspect they would be used to the false case-insensitive match. I
tested it on Windows XP with NTFS: İ (U+0130) did not match I or i
or ı (U+0131). I also tested it with the Turkish language/keyboard set.

Once it's set in a filesystem, the handling of it can't really be
swapped back and forth either, otherwise, you may lose access to
that file.

There is no practical way that I can see of supporting this
fully, even with using the NLS tables. The on-disk hashes have
to remain consistent regardless of what language is specified.

Barry.
-
To unsubscribe from this list: send the line unsubscribe linux-fsdevel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: msync(2) bug(?), returns AOP_WRITEPAGE_ACTIVATE to userland

2007-10-08 Thread Pekka Enberg
Hi Ryan,

On 10/8/07, Ryan Finnie [EMAIL PROTECTED] wrote:
 Doesn't appear to be enough.  I can't figure out why (since it appears
 write_cache_pages bubbles up directly to sys_msync), but with that
 patch applied, in my test case[1], msync returns -1 EIO.  However,
 with the exact same kernel without that patch applied, msync returns
 524288 (AOP_WRITEPAGE_ACTIVATE).  But as your patch specifically flips
 524288 to 0, I can't figure out how it eventually returns  -1 EIO.

 [1] apt-get check on a unionfs2 mount backed by tmpfs over cdrom,
 standard livecd setup

You have swap device disabled, right? If so, I can't see any reason
why msync(2) on tmpfs would return -EIO. Can you please send a strace
log for your test case?

   Pekka
-
To unsubscribe from this list: send the line unsubscribe linux-fsdevel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH]fix VM_CAN_NONLINEAR check in sys_remap_file_pages

2007-10-08 Thread Yan Zheng
Hi all

The test for VM_CAN_NONLINEAR always fails

Signed-off-by: Yan Zheng[EMAIL PROTECTED]

diff -ur linux-2.6.23-rc9/mm/fremap.c linux/mm/fremap.c
--- linux-2.6.23-rc9/mm/fremap.c2007-10-07 15:03:33.0 +0800
+++ linux/mm/fremap.c   2007-10-08 19:33:44.0 +0800
@@ -160,7 +160,7 @@
if (vma-vm_private_data  !(vma-vm_flags  VM_NONLINEAR))
goto out;

-   if (!vma-vm_flags  VM_CAN_NONLINEAR)
+   if (!(vma-vm_flags  VM_CAN_NONLINEAR))
goto out;

if (end = start || start  vma-vm_start || end  vma-vm_end)
-
To unsubscribe from this list: send the line unsubscribe linux-fsdevel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH]fix VM_CAN_NONLINEAR check in sys_remap_file_pages

2007-10-08 Thread Andrew Morton
On Mon, 8 Oct 2007 19:45:08 +0800 Yan Zheng [EMAIL PROTECTED] wrote:

 Hi all
 
 The test for VM_CAN_NONLINEAR always fails
 
 Signed-off-by: Yan Zheng[EMAIL PROTECTED]
 
 diff -ur linux-2.6.23-rc9/mm/fremap.c linux/mm/fremap.c
 --- linux-2.6.23-rc9/mm/fremap.c  2007-10-07 15:03:33.0 +0800
 +++ linux/mm/fremap.c 2007-10-08 19:33:44.0 +0800
 @@ -160,7 +160,7 @@
   if (vma-vm_private_data  !(vma-vm_flags  VM_NONLINEAR))
   goto out;
 
 - if (!vma-vm_flags  VM_CAN_NONLINEAR)
 + if (!(vma-vm_flags  VM_CAN_NONLINEAR))
   goto out;
 
   if (end = start || start  vma-vm_start || end  vma-vm_end)

Lovely.  From this we can deduce that nobody has run remap_file_pages() since
2.6.23-rc1 and that nobody (including the developer who made that change) ran it
while that change was in -mm.

I'm surprise that LTP doesn't have any remap_file_pages() tests.

Have you runtime tested this change?

Thanks.
-
To unsubscribe from this list: send the line unsubscribe linux-fsdevel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH]fix VM_CAN_NONLINEAR check in sys_remap_file_pages

2007-10-08 Thread Randy Dunlap
On Mon, 8 Oct 2007 10:04:56 -0700 Andrew Morton wrote:

 On Mon, 8 Oct 2007 19:45:08 +0800 Yan Zheng [EMAIL PROTECTED] wrote:
 
  Hi all
  
  The test for VM_CAN_NONLINEAR always fails
  
  Signed-off-by: Yan Zheng[EMAIL PROTECTED]
  
  diff -ur linux-2.6.23-rc9/mm/fremap.c linux/mm/fremap.c
  --- linux-2.6.23-rc9/mm/fremap.c2007-10-07 15:03:33.0 +0800
  +++ linux/mm/fremap.c   2007-10-08 19:33:44.0 +0800
  @@ -160,7 +160,7 @@
  if (vma-vm_private_data  !(vma-vm_flags  VM_NONLINEAR))
  goto out;
  
  -   if (!vma-vm_flags  VM_CAN_NONLINEAR)
  +   if (!(vma-vm_flags  VM_CAN_NONLINEAR))
  goto out;
  
  if (end = start || start  vma-vm_start || end  vma-vm_end)
 
 Lovely.  From this we can deduce that nobody has run remap_file_pages() since
 2.6.23-rc1 and that nobody (including the developer who made that change) ran 
 it
 while that change was in -mm.

I've run rmap-test with -M (use remap_file_pages) and
remap-test from ext3-tools, but not remap_file_pages for some reason.

I'll now add remap_file_pages soon.
Maybe those other 2 tests aren't strong enough (?).
Or maybe they don't return a non-0 exit status even when they fail...
(I'll check.)


 I'm surprise that LTP doesn't have any remap_file_pages() tests.

quick grep didn't find any for me.

 Have you runtime tested this change?
 
 Thanks.


---
~Randy
-
To unsubscribe from this list: send the line unsubscribe linux-fsdevel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH]fix VM_CAN_NONLINEAR check in sys_remap_file_pages

2007-10-08 Thread Andrew Morton
On Mon, 8 Oct 2007 10:28:43 -0700
Randy Dunlap [EMAIL PROTECTED] wrote:

 On Mon, 8 Oct 2007 10:04:56 -0700 Andrew Morton wrote:
 
  On Mon, 8 Oct 2007 19:45:08 +0800 Yan Zheng [EMAIL PROTECTED] wrote:
  
   Hi all
   
   The test for VM_CAN_NONLINEAR always fails
   
   Signed-off-by: Yan Zheng[EMAIL PROTECTED]
   
   diff -ur linux-2.6.23-rc9/mm/fremap.c linux/mm/fremap.c
   --- linux-2.6.23-rc9/mm/fremap.c  2007-10-07 15:03:33.0 +0800
   +++ linux/mm/fremap.c 2007-10-08 19:33:44.0 +0800
   @@ -160,7 +160,7 @@
 if (vma-vm_private_data  !(vma-vm_flags  VM_NONLINEAR))
 goto out;
   
   - if (!vma-vm_flags  VM_CAN_NONLINEAR)
   + if (!(vma-vm_flags  VM_CAN_NONLINEAR))
 goto out;
   
 if (end = start || start  vma-vm_start || end  vma-vm_end)
  
  Lovely.  From this we can deduce that nobody has run remap_file_pages() 
  since
  2.6.23-rc1 and that nobody (including the developer who made that change) 
  ran it
  while that change was in -mm.
 
 I've run rmap-test with -M (use remap_file_pages) and
 remap-test from ext3-tools, but not remap_file_pages for some reason.
 
 I'll now add remap_file_pages soon.
 Maybe those other 2 tests aren't strong enough (?).
 Or maybe they don't return a non-0 exit status even when they fail...
 (I'll check.)

Perhaps Yan Zheng can tell us what test was used to demonstrate this?

 
  I'm surprise that LTP doesn't have any remap_file_pages() tests.
 
 quick grep didn't find any for me.

Me either.  There are a few lying around the place which could be
integrated.

It would be good if LTP were to have some remap_file_pages() tests
(please).  As we see here, it is something which we can easily break, and
leave broken for some time.

-
To unsubscribe from this list: send the line unsubscribe linux-fsdevel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH]fix VM_CAN_NONLINEAR check in sys_remap_file_pages

2007-10-08 Thread Hugh Dickins
On Mon, 8 Oct 2007, Yan Zheng wrote:
 
 The test for VM_CAN_NONLINEAR always fails

Good catch indeed.  Though I was puzzled how we do nonlinear at all,
until I realized it's The test for not VM_CAN_NONLINEAR always fails.

It's not as serious as it appears, since code further down has been
added more recently to simulate nonlinear on non-RAM-backed filesystems,
instead of going the real nonlinear way; so most filesystems are now not
required to do what VM_CAN_NONLINEAR was put in to ensure they could do.

I'm confused as to where that leaves us: is this actually a fix that
needs to go into 2.6.23?  or will it suddenly disable a system call
which has been silently working fine on various filesystems which did
not add VM_CAN_NONLINEAR?  could we just rip out VM_CAN_NONLINEAR?
I hope Nick or Miklos is clearer on what the risks are.

(Apologies for all the nots and nons here, I'm embarrassed
after just criticizing Ingo's SCHED_NO_NO_OMIT_FRAME_POINTER!)

Hugh

 
 Signed-off-by: Yan Zheng[EMAIL PROTECTED]
 
 diff -ur linux-2.6.23-rc9/mm/fremap.c linux/mm/fremap.c
 --- linux-2.6.23-rc9/mm/fremap.c  2007-10-07 15:03:33.0 +0800
 +++ linux/mm/fremap.c 2007-10-08 19:33:44.0 +0800
 @@ -160,7 +160,7 @@
   if (vma-vm_private_data  !(vma-vm_flags  VM_NONLINEAR))
   goto out;
 
 - if (!vma-vm_flags  VM_CAN_NONLINEAR)
 + if (!(vma-vm_flags  VM_CAN_NONLINEAR))
   goto out;
 
   if (end = start || start  vma-vm_start || end  vma-vm_end)
 -
 To unsubscribe from this list: send the line unsubscribe linux-kernel in
 the body of a message to [EMAIL PROTECTED]
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
 Please read the FAQ at  http://www.tux.org/lkml/
 
-
To unsubscribe from this list: send the line unsubscribe linux-fsdevel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH]fix VM_CAN_NONLINEAR check in sys_remap_file_pages

2007-10-08 Thread Yan Zheng
2007/10/8, Hugh Dickins [EMAIL PROTECTED]:
 On Mon, 8 Oct 2007, Yan Zheng wrote:
 
  The test for VM_CAN_NONLINEAR always fails
 Good catch indeed.  Though I was puzzled how we do nonlinear at all,
 until I realized it's The test for not VM_CAN_NONLINEAR always fails.
 It's not as serious as it appears, since code further down has been
 added more recently to simulate nonlinear on non-RAM-backed filesystems,
 instead of going the real nonlinear way; so most filesystems are now not
 required to do what VM_CAN_NONLINEAR was put in to ensure they could do.
 I'm confused as to where that leaves us: is this actually a fix that
 needs to go into 2.6.23?  or will it suddenly disable a system call
 which has been silently working fine on various filesystems which did
 not add VM_CAN_NONLINEAR?  could we just rip out VM_CAN_NONLINEAR?
 I hope Nick or Miklos is clearer on what the risks are.
 (Apologies for all the nots and nons here, I'm embarrassed
 after just criticizing Ingo's SCHED_NO_NO_OMIT_FRAME_POINTER!)
 Hugh

Yes, I mean The test for not VM_CAN_NONLINEAR always fails.  please
forgive my poor English.
-
To unsubscribe from this list: send the line unsubscribe linux-fsdevel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH]fix VM_CAN_NONLINEAR check in sys_remap_file_pages

2007-10-08 Thread Nick Piggin
On Monday 08 October 2007 23:37, Hugh Dickins wrote:
 On Mon, 8 Oct 2007, Yan Zheng wrote:
  The test for VM_CAN_NONLINEAR always fails

 Good catch indeed.  Though I was puzzled how we do nonlinear at all,
 until I realized it's The test for not VM_CAN_NONLINEAR always fails.

 It's not as serious as it appears, since code further down has been
 added more recently to simulate nonlinear on non-RAM-backed filesystems,
 instead of going the real nonlinear way; so most filesystems are now not
 required to do what VM_CAN_NONLINEAR was put in to ensure they could do.

Well, I think all filesystems can do VM_CAN_NONLINEAR anyway. Device
drivers and weird things tend to have trouble...


 I'm confused as to where that leaves us: is this actually a fix that
 needs to go into 2.6.23?  or will it suddenly disable a system call
 which has been silently working fine on various filesystems which did
 not add VM_CAN_NONLINEAR?  could we just rip out VM_CAN_NONLINEAR?

We probably should keep VM_CAN_NONLINEAR for the moment, I think.
But now that we have the fallback path, we _could_ use that instead of
failing. I doubt anybody will be using nonlinear mappings on anything but
regular files for the time being, but as a trivial fix, I think this probably
should go into 2.6.23.

Thanks for spotting this problem
Acked-by: Nick Piggin [EMAIL PROTECTED]

 I hope Nick or Miklos is clearer on what the risks are.

 (Apologies for all the nots and nons here, I'm embarrassed
 after just criticizing Ingo's SCHED_NO_NO_OMIT_FRAME_POINTER!)

 Hugh

  Signed-off-by: Yan Zheng[EMAIL PROTECTED]
  
  diff -ur linux-2.6.23-rc9/mm/fremap.c linux/mm/fremap.c
  --- linux-2.6.23-rc9/mm/fremap.c2007-10-07 15:03:33.0 +0800
  +++ linux/mm/fremap.c   2007-10-08 19:33:44.0 +0800
  @@ -160,7 +160,7 @@
  if (vma-vm_private_data  !(vma-vm_flags  VM_NONLINEAR))
  goto out;
 
  -   if (!vma-vm_flags  VM_CAN_NONLINEAR)
  +   if (!(vma-vm_flags  VM_CAN_NONLINEAR))
  goto out;
 
  if (end = start || start  vma-vm_start || end  vma-vm_end)
  -
  To unsubscribe from this list: send the line unsubscribe linux-kernel
  in the body of a message to [EMAIL PROTECTED]
  More majordomo info at  http://vger.kernel.org/majordomo-info.html
  Please read the FAQ at  http://www.tux.org/lkml/
-
To unsubscribe from this list: send the line unsubscribe linux-fsdevel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH]fix VM_CAN_NONLINEAR check in sys_remap_file_pages

2007-10-08 Thread Nick Piggin
On Tuesday 09 October 2007 03:04, Andrew Morton wrote:
 On Mon, 8 Oct 2007 19:45:08 +0800 Yan Zheng [EMAIL PROTECTED] wrote:
  Hi all
 
  The test for VM_CAN_NONLINEAR always fails
 
  Signed-off-by: Yan Zheng[EMAIL PROTECTED]
  
  diff -ur linux-2.6.23-rc9/mm/fremap.c linux/mm/fremap.c
  --- linux-2.6.23-rc9/mm/fremap.c2007-10-07 15:03:33.0 +0800
  +++ linux/mm/fremap.c   2007-10-08 19:33:44.0 +0800
  @@ -160,7 +160,7 @@
  if (vma-vm_private_data  !(vma-vm_flags  VM_NONLINEAR))
  goto out;
 
  -   if (!vma-vm_flags  VM_CAN_NONLINEAR)
  +   if (!(vma-vm_flags  VM_CAN_NONLINEAR))
  goto out;
 
  if (end = start || start  vma-vm_start || end  vma-vm_end)

 Lovely.  From this we can deduce that nobody has run remap_file_pages()
 since 2.6.23-rc1 and that nobody (including the developer who made that
 change) ran it while that change was in -mm.

But you'd be wrong. remap_file_pages was tested both with my own tester
and Ingo's test program.

vm_flags != 0, !vm_flags = 0, 0  x = 0, so the test always falls
through. Of course, what I _should_ have done is also test a driver which
does not have VM_CAN_NONLINEAR... but even I wouldn't rewrite half
the nonlinear mapping code without once testing it ;)

FWIW, Oracle (maybe the sole real user of this) has been testing it, which
I'm very happy about (rather than testing after 2.6.23 is released).
-
To unsubscribe from this list: send the line unsubscribe linux-fsdevel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH]fix VM_CAN_NONLINEAR check in sys_remap_file_pages

2007-10-08 Thread Nick Piggin
On Tuesday 09 October 2007 03:51, Andrew Morton wrote:
 On Mon, 8 Oct 2007 10:28:43 -0700

  I'll now add remap_file_pages soon.
  Maybe those other 2 tests aren't strong enough (?).
  Or maybe they don't return a non-0 exit status even when they fail...
  (I'll check.)

 Perhaps Yan Zheng can tell us what test was used to demonstrate this?

Was probably found by review. Otherwise, you could probably reproduce
it by mmaping, say, drm device node, running remap_file_pages() on it
to create a nonlinear mapping, and then finding that you get the wrong
data.

   I'm surprise that LTP doesn't have any remap_file_pages() tests.
 
  quick grep didn't find any for me.

 Me either.  There are a few lying around the place which could be
 integrated.

 It would be good if LTP were to have some remap_file_pages() tests
 (please).  As we see here, it is something which we can easily break, and
 leave broken for some time.

Here is Ingo's old test, since cleaned up and fixed a bit by me
I'm sure he would distribute it GPL, but I've cc'ed him because I didn't
find an explicit statement about that.

/*
 * Copyright (C) Ingo Molnar, 2002
 */
#define _GNU_SOURCE
#include stdio.h
#include unistd.h
#include sys/mman.h
#include sys/stat.h
#include sys/types.h
#include fcntl.h
#include errno.h
#include stdlib.h
#include sys/times.h
#include sys/wait.h
#include sys/ioctl.h
#include sys/syscall.h
#include linux/unistd.h

#define PAGE_SIZE 4096
#define PAGE_WORDS (PAGE_SIZE/sizeof(int))

#define CACHE_PAGES 1024
#define CACHE_SIZE (CACHE_PAGES*PAGE_SIZE)

#define WINDOW_PAGES 16
#define WINDOW_SIZE (WINDOW_PAGES*PAGE_SIZE)

#define WINDOW_START 0x4800

static char cache_contents [CACHE_SIZE];

static void test_nonlinear(int fd)
{
	char *data = NULL;
	int i, j, repeat = 2;

	for (i = 0; i  CACHE_PAGES; i++) {
		int *page = (int *) (cache_contents + i*PAGE_SIZE);

		for (j = 0; j  PAGE_WORDS; j++)
			page[j] = i;
	}

	if (write(fd, cache_contents, CACHE_SIZE) != CACHE_SIZE)
		perror(write), exit(1);

	data = mmap((void *)WINDOW_START,
			WINDOW_SIZE,
			PROT_READ|PROT_WRITE, 
			MAP_FIXED | MAP_SHARED 
			, fd, 0);

	if (data == MAP_FAILED)
		perror(mmap), exit(1);

again:
	for (i = 0; i  WINDOW_PAGES; i += 2) {
		char *page = data + i*PAGE_SIZE;

		if (remap_file_pages(page, PAGE_SIZE * 2, 0,
(WINDOW_PAGES-i-2), 0) == -1)
			perror(remap_file_pages), exit(1);
	}

	for (i = 0; i  WINDOW_PAGES; i++) {
		/*
		 * Double-check the correctness of the mapping:
		 */
		if (i  1) {
			if (data[i*PAGE_SIZE] != WINDOW_PAGES-i) {
printf(hm, mapped incorrect data!\n);
exit(1);
			}
		} else {
			if (data[i*PAGE_SIZE] != WINDOW_PAGES-i-2) {
printf(hm, mapped incorrect data!\n);
exit(1);
			}
		}
	}

	if (--repeat)
		goto again;
}

int main(int argc, char **argv)
{
	int fd;

	fd = open(/dev/shm/cache, O_RDWR|O_CREAT|O_TRUNC,S_IRWXU);
	if (fd  0)
		perror(open), exit(1);
	test_nonlinear(fd);
	if (close(fd) == -1)
		perror(close), exit(1);
	printf(nonlinear shm file OK\n);

	fd = open(/tmp/cache, O_RDWR|O_CREAT|O_TRUNC,S_IRWXU);
	if (fd  0)
		perror(open), exit(1);
	test_nonlinear(fd);
	if (close(fd) == -1)
		perror(close), exit(1);
	printf(nonlinear /tmp/ file OK\n);

	exit(0);
}



Re: [PATCH]fix VM_CAN_NONLINEAR check in sys_remap_file_pages

2007-10-08 Thread Yan Zheng
2007/10/9, Andrew Morton [EMAIL PROTECTED]:
 Perhaps Yan Zheng can tell us what test was used to demonstrate this?

I found it by review, only do test to check remap_file_pages works
when VM_CAN_NONLINEAR flags is set.
-
To unsubscribe from this list: send the line unsubscribe linux-fsdevel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html