+1000000000000000000000 On 12/16/05, John Davis <[EMAIL PROTECTED]> wrote: > If one uses the file offset fields in the OVERLAPPED struct and passes > this to Read/WriteFile then there is no need to call SetFilePointer, and > we have an atomic operation. As a result the race condition between > SetFilePointer and Read/WriteFile is eliminated. Similar methods should > be exposed in the framework. > > On Fri, 16 Dec 2005 09:57:10 -0500, Peter Ritchie > <[EMAIL PROTECTED]> wrote: > > >Davis, > >> > >>In OVERLAPPED there is an Offset and an OffsetHight member. These are > not > >>reserved. And your case for SetFileEx is just flat wrong. > > > >Sorry, I mis-read the internal-use-only for Offset and OffsetHigh. From > >the documentation for SetFilePointer > >http://msdn.microsoft.com/library/default.asp?url=/library/en- > >us/fileio/fs/setfilepointer.asp (http://tinyurl.com/9lpvn): > >"Be careful when you set a file pointer in a multi-threaded application. > >You must synchronize access to shared resources. For example, an > >application with threads that share a file handle, update the file > >pointer, and read from the file must protect this sequence by using a > >critical section object or mutex object." > >Sounds pretty much the same as what I was saying (admitting my mistake > >regarding the two members of OVERLAPPED). > > > >> > >>I'm guessing you haven't done much async file programming. Using IO > >>completion ports or overlapped io through this struct can dramatically > >>increase throughput for an app. This is due primarily to the fact that > >the > >>io scheduler on the hard drive (or raid array, etc.) can actually handle > >>multiple reads/writes at the same time. Having multiple outstanding io > >>requests at a time is much more efficient than only doing one at a time. > >> > > > >I've been programming IO asynchronously (not the same as multi-threaded > >IO) for years (not with disk drives, through native WriteFile/ReadFile > >with high-latency devices)... > > > >>The point is to not have your app block on a single IO operation. > Instead > >>of having 10 threads doing reads/writes one can use > WaitForMultipleObjects > >>or GetQueuedCompletion status to receive IO notifications on a single > >>thread. > > > >That was my point, and that's what BeginRead/BeginWrite do: use WriteFile > >with an overlapped structure. Asynchronous IO is different then multi- > >threaded IO. I wasn't suggesting that overlapped IO should be avoided or > >that it doesn't improve the responsiveness of an application. Quite the > >contrary, my overlapped IO apps would be horribly unresponsive if I didn't > >use overlapped IO. In the general case (regardless of whether it's IO) > >adding more than a background thread to an GUI application does not > >increase performance (it may decrease performance depending on the non- > >.NET runtime libraries you're using--I can't comment on the .NET runtime, > >in this case). I do advocate avoiding doing anything but UI logic on the > >UI thread; but, that's usually just a case of adding a background thread > >and/or using overlapped IO (in my overlapped IO code, I use a background > >thread to perform the read/write; but, again, I'm using high-latency > >devices.) > > > >RAID doesn't imply that the array can handle multiple accesses at the same > >time; it has a queue as any other (useful) mass-storage device does. Two > >threads waiting for the device's queue/cache to finalize their > >asynchronous file access isn't going to be any more responsive than an a > >single background thread waiting for two asynchronous file accesses. > > > >I guess the question is, why is asynchronous IO with FileStream.BeginWrite > >()/BeginRead() not good enough for your purposes? If you want to do > multi- > >threaded access with FileStream you're either going to have to synchronize > >access to a single FileStream object, or use one FileStream object per > >thread. This is exactly how FileStream is documented; there's no bug. > > > >If you want a thread-safe object to perform multi-threaded asynchronous IO > >on the same file handle you'll have to write your own; that's not what > >FileStream is intended for. > > > >http://www.peterRitchie.com/ > > > >On Thu, 15 Dec 2005 13:29:49 -0600, John Davis <[EMAIL PROTECTED]> wrote: > > > >>See: > >>http://msdn.microsoft.com/library/default.asp?url=/library/en- > >us/dllproc/base/overlapped_str.asp > >>(watch the wrap on the url) > >> > >>Ritchie, > > > > > >> > >>>From: Peter Ritchie <[EMAIL PROTECTED]> > >>>Reply-To: "Discussion of advanced .NET topics." > >>><[email protected]> > >>>To: [email protected] > >>>Subject: Re: [ADVANCED-DOTNET] Possible defect in BeginWrite on > >FileStream > >>>Date: Thu, 15 Dec 2005 12:10:18 -0500 > >>> > >>>BTW, you're not supposed to use the members of OVERLAPPED (other than > >>>hEvent during creation). You can certainly read them, for whatever it's > >>>worth; but the remaining four members are documented as "Reserved for > >>>operating system use.". So, you can't get what you want with WriteFile > >>>and the OVERLAPPED struct. You're supposed to use SetFilePointer[Ex] to > >>>seek to a position in a file and begin writing. And, if it isn't > >obvious, > >>>there's only one file pointer to file: multi-threaded applications must > >>>synchronize access to the file. > >>> > >>>What do you think multi-threaded writing to a file actually buys you? > >>>There's only one head on the hard drive that can perform the write. > Only > >>>one thread is really going to be writing at a time anyway; even on a > >multi- > >>>processor computer. You might be able to get two threads to write the > >>>cache at the same time with a multi-processor computer (you'd have to > >have > >>>at least 4 processors; the main processor is usually busy time-slicing > >the > >>>majority of operations); but, again, what does that do for you? > >>> > >>>Overlapped IO is nice because your GUI thread can be responsive while a > >>>potentially long read/write operation is done "on the background"; but, > >it > >>>doesn't imply multi-threaded access to a file. > >>> > >>>http://www.peterRitchie.com/ > >>> > >>> > >>>On Thu, 15 Dec 2005 10:01:39 -0500, John Davis <[EMAIL PROTECTED]> > wrote: > >>> > >>> >Ian I'm afraid you're really mistaken on this one. > >>> > > >>> >What I'm looking for is the equivalent of a ReadFile or WriteFile call > >>> >where the offset is passed in the OVERLAPPED struct. The call can > >occur > >>> >through overlapped IO, or an IO completion port. The call is also > >>>atomic. > >>> > > >>> >There needs to be a random access async read/write of some sort which > >>>uses > >>> >the OVERLAPPED struct in the Win32 API in that it has a file offset > >field > >>> >for the read or write. If one observes the current implementation in > >>> >DotNetReflector, the OVERLAPPED struct is getting populated via the > >>> >FileStream Seek call. The OVERLAPPED struct is then used on the next > >>> >async write. Why implement it this way if FileStreams can only be > used > >>>by > >>> >a single thread? This is a bit of a cluster if you ask me. > >>> > > >>> >On Thu, 26 May 2005 23:22:45 +0100, Ian Griffiths <[EMAIL PROTECTED] > >SW.CO.UK> > >>> >wrote: > >>> > > >>> >>Even if there were an atomic method how would that help? It wouldn't > >>> >>eliminate the race condition, because it would still be illegal to > >call > >>> >>the method simultaneously on multiple threads. (FileStream's > >>> >>documentation says it's illegal to do that.) So you'd still need to > >use > >>> >>locking or some other synchronization method to make sure your > threads > >>> >>take it in turns even if there were an atomic method. > >>> >> > >>> >>I agree it would be cleaner to have a random access method if random > >>> >>access is what you're doing - a stream isn't really the right > >>> >>abstraction in that case. (Sadly it's the only one we're offered > >>> >>AFAIK.) > >>> >> > >>> >>But is this a defect? It's a bug to use a stream from multiple > >threads > >>> >>without synchronization. It would still be a bug even with the > atomic > >>> >>method you propose. > >>> >> > >>> >> > >>> >>-- > >>> >>Ian Griffiths > >>> >>http://www.interact-sw.co.uk/iangblog/ > >>> >> > >>> >>> -----Original Message----- > >>> >>> From: John Davis > >>> >>> > >>> >>> >Are you saying that locking, Seeking, BeginWriteing, and unlocking > >is > >>> >>> not good enough? > >>> >>> > >>> >>> Well, I guess it is good enough, just doesn't seem very elegant. > >>> >>Would be > >>> >>> nice if there was an offset in there so one "atomic" call could be > >>> >>made. > >>> >>> Same as with Win32 overlapped io. > >>> >>> > >>> >>> >Does the BeginWrite not in effect take a snapshot of > >>> >>> the seek location when the operation starts? > >>> >>> > >>> >>> I don't know for certain if it does, I certainly hope so. Time to > >go > >>> >>> digging with reflector I guess. > >>> >>> > >>> >>> Ian Griffiths <[EMAIL PROTECTED]> wrote: > >>> >>> The documentation says that members of this class are not thread > >safe, > >>> >>> so multiple threads shouldn't be using this at any given instance. > >You > >>> >>> need to do some kind of locking if you have multiple threads using > >it. > >>> >>> > >>> >>> Are you saying that locking, Seeking, BeginWriteing, and unlocking > >is > >>> >>> not good enough? Does the BeginWrite not in effect take a snapshot > >of > >>> >>> the seek location when the operation starts? > >>> >>> > >>> >>> > >>> >>> > -----Original Message----- > >>> >>> > From: John Davis > >>> >>> > > >>> >>> > The way I understand it, the current BeginWrite on the > >>> >>> > FileStream object will write out it's buffer at the location of > >>> >>> > the last Seek. The problem comes when there are multiple threads > >>> >>> > using the same FileStream instance. > >>> >>> > > >>> >>> > Each one is looking to write to a different offset in the file. > >This > >>> >>> > means each thread must call FileStream.Seek followed by > >>> >>> > FileStream.BeginWrite Obviously, this creates a race condition. > >>> >>> > > >>> >>> > Why isn't there an async BeginWrite, on the FileStream, with an > >>> >>> > argument which specifies the offset within the file to write to. > >>> >>> > It's possible to call WriteFileEx within the Win32 API with an > >>> >>> > OVERLAPPED struct which specifies the offset. Why isn't this > >>> >>> > exposed through the framework? > >>> >> > >>> >>=================================== > >>> >>This list is hosted by DevelopMentor� http://www.develop.com > >>> >> > >>> >>View archives and manage your subscription(s) at > >>> >http://discuss.develop.com > >>> > > >>> >There needs to be a random access async read/write of some sort which > >>>uses > >>> >the OVERLAPPED struct in the Win32 API in that it has a file offset > >field > >>> >for the read or write. If one observes the current implementation in > >>> >DotNetReflector, the OVERLAPPED struct is getting populated via the > >>> >FileStream Seek call. The OVERLAPPED struct is then used on the next > >>> >async write. Why implement it this way if FileStreams can only be > used > >>>by > >>> >a single thread? This is a bit of a cluster if you ask me. > >>> > >>>=================================== > >>>This list is hosted by DevelopMentor� http://www.develop.com > >>> > >>>View archives and manage your subscription(s) at > >http://discuss.develop.com > >> > >>=================================== > >>This list is hosted by DevelopMentor� http://www.develop.com > >> > >>View archives and manage your subscription(s) at > >http://discuss.develop.com > > > >s > > > >=================================== > >This list is hosted by DevelopMentor� http://www.develop.com > > > >View archives and manage your subscription(s) at > http://discuss.develop.com > > =================================== > This list is hosted by DevelopMentor� http://www.develop.com > > View archives and manage your subscription(s) at http://discuss.develop.com >
-- Cheers, Stoyan
