> Chris, > > > We could translate those errors into EIO or > something > > documented by the msync man page, but that's less > > descriptive and potentially confusing. Perhaps we > > should consider updating the man page. > > As a programmer I would much prefer to have the most > descriptive errors possible. > > The thing that surprises me is that i get an error > that seems to indicate that "I can't use any > MORE space". I am not trying to use any MORE space, I > am just trying to overwrite the space > that I have already been allocated from the file > system. If I had created a file with holes in it then > I > could understand a msync generating an error when it > tried to fill the holes. Although perhaps the > mmap should have generated the error not the msync . > . . not sure. > > It seems to me that this is a backwards > incomparability. My program goes to a great deal of > effort > to deal with running out of disk space by checking > all the writes, and dealing with errors. But it > does very little checking on msync. The last thing I > expected was to find that the disk space that > had apparently already been allocated to me through > writes had been mysteriously unallocated behind my > back.
Is there a snapshot in effect? If so, a write means the old blocks (as seen in the snapshot) have to be retained as well as the new ones; even if the file isn't growing, the amount of space it uses grows. That could cause all sorts of syscalls to get ENOSPC or EDQUOT that don't on other filesystems (and probably very rarely (give or take _very_ tight space) do on zfs if there is no snapshot holding the old blocks). > I just had a thought. If I had not done the msync but > had instead just exited my program would my > changes to the mmap region have been written to disk > by fsflush or would fsflush have failed to > write my changes? If fsflush failed to write the > changes would it have generated/logged an error? Seems to me that if you couldn't do it, fsflush (or whatever; maybe zfs has its own mechanism?) wouldn't be able to either. No idea if that sort of thing would get logged; even if it should, zfs adds situations where such things could occur that might not previously have arisen, so that might take either an expert or some major code-diving to figure out. Of course, if you don't use msync()/fsync() (or with write(), some suitable combination of the O_*SYNC flags), your program would never really have any way of knowing whether your write succeeded (could be an I/O error on any fs type, if the device went bad). The possibility of heretofore unexpected ENOSPC/EDQUOT just means you have one more reason to do what you always should have done. > I had another thought. If I had fsynced rather than > msynced would I have got an error? I wouldn't mix write()/fsync() with mmap()/msync(); it doesn't necessarily work on all systems, and I think zfs sidesteps the usual page cache; it may still provide the same consistency between the two that other filesystems have historically provided on Solaris, but perhaps at some cost. At the very least, if that's not a best practice, you may be on a code path less often taken, and thus less exercised. :-) > Note: I modified my program so that it didn't use > mmap. Instead after I had filled up the > disk using the second file I then tried to overwrite > the contents of the first using pwrite. > These attempts to overwrite using pwrite also failed > with EDQUOT. This just seems wrong. > I'm not trying to get more disk space, i'm just > trying to change the contents of the space > that is already mine. Same situation as with the msync(), I think; namely that any sort of write (even to the inode, as in an fchmod() as mentioned in another thread) could fail under certain circumstances with ENOSPC/EDQUOT. Short of a new option to have snapshots get their own private quotas/reservations, I don't see how one could have all the Good Things zfs offers and not have some such effect. -- This messages posted from opensolaris.org
