Re: Disk IO - coss and ufs with common IO engines.

2003-09-01 Thread Adrian Chadd 
On Sun, Aug 31, 2003, Robert Collins wrote:

  If you want to use the async io code rom anywhere, say, logfile writing,
  you're _going_ to have to do write combining or your write queue is going
  to quickly blow out to a stupid size. The store IO interface really assures
  us there's only one outstanding write per object at a time, unless
  something whacked has change which I haven't yet picked up on.
 
 This isn't the store IO interface. Each DiskFile can have multiple
 outstanding requests, both read or write. This is a requirement of COSS.

Ah, yes, of course it'll have to.

 Nope. That is a user-of-the-interface problem, it's not appropriate to
 force the interface to police this.
 
 We can certainly provide a write-combining Decorator though.
 Conceptually such a layer would:
 wrap the IOStrategy to return a decorated instance.
 wrap the DiskFile objects to intercept write's.
 Queue writes per File in memory, and when callback() is called, flush
 the queue to the real objects.
 
 Maybe 200 lines of code for the interested.

*nod* it'll still have to be done, else your write queues will start
filling and very quickly.



adrian

Re: Disk IO - coss and ufs with common IO engines.

2003-08-31 Thread Adrian Chadd 
On Sat, Aug 30, 2003, Serassio Guido wrote:

 changes that can be improved upon, etc.
 
 This is a very interesting thing:
 
 For example it means that a Windows native awin32 fs based on aufs is no 
 more needed: all the magic can will be do from a Win32DiskThreads.


Or, once COSS has been fixed up a little more, a native COSS fs
which can use the win32 disk io code.

Which I'm almost certain will perform well. :-P



Adrian

Re: Disk IO - coss and ufs with common IO engines.

2003-08-31 Thread Robert Collins 
On Sun, 2003-08-31 at 12:08, Adrian Chadd wrote:
 On Sat, Aug 30, 2003, Robert Collins wrote:
 
  The only step missing to be able to use the IO modules from anywhere is
  automatic registration to be 'synced' at shutdown, and 'callback()'
  checked on each io loop.
 
 If you want to use the async io code rom anywhere, say, logfile writing,
 you're _going_ to have to do write combining or your write queue is going
 to quickly blow out to a stupid size. The store IO interface really assures
 us there's only one outstanding write per object at a time, unless
 something whacked has change which I haven't yet picked up on.

This isn't the store IO interface. Each DiskFile can have multiple
outstanding requests, both read or write. This is a requirement of COSS.

 Funnily enough, if you check out src/disk.c, you'll find some stuff which
 was done many, many, many moons ago to deal with async write combining.

Yah, long term I'd like to push disk.c into the DiskIO infrastructure,
and remove the common file_* functions completely.

 Just 2c. Yes, it'd be nice to have true async writes for logfiles and such.
 But you're going to have to do write combining.

Nope. That is a user-of-the-interface problem, it's not appropriate to
force the interface to police this.

We can certainly provide a write-combining Decorator though.
Conceptually such a layer would:
wrap the IOStrategy to return a decorated instance.
wrap the DiskFile objects to intercept write's.
Queue writes per File in memory, and when callback() is called, flush
the queue to the real objects.

Maybe 200 lines of code for the interested.

Cheers,
Rob

-- 
GPG key available at: http://members.aardvark.net.au/lifeless/keys.txt.


signature.asc
Description: This is a digitally signed message part