Davis, > >In OVERLAPPED there is an Offset and an OffsetHight member. These are not >reserved. And your case for SetFileEx is just flat wrong.
Sorry, I mis-read the internal-use-only for Offset and OffsetHigh. From the documentation for SetFilePointer http://msdn.microsoft.com/library/default.asp?url=/library/en- us/fileio/fs/setfilepointer.asp (http://tinyurl.com/9lpvn): "Be careful when you set a file pointer in a multi-threaded application. You must synchronize access to shared resources. For example, an application with threads that share a file handle, update the file pointer, and read from the file must protect this sequence by using a critical section object or mutex object." Sounds pretty much the same as what I was saying (admitting my mistake regarding the two members of OVERLAPPED). > >I'm guessing you haven't done much async file programming. Using IO >completion ports or overlapped io through this struct can dramatically >increase throughput for an app. This is due primarily to the fact that the >io scheduler on the hard drive (or raid array, etc.) can actually handle >multiple reads/writes at the same time. Having multiple outstanding io >requests at a time is much more efficient than only doing one at a time. > I've been programming IO asynchronously (not the same as multi-threaded IO) for years (not with disk drives, through native WriteFile/ReadFile with high-latency devices)... >The point is to not have your app block on a single IO operation. Instead >of having 10 threads doing reads/writes one can use WaitForMultipleObjects >or GetQueuedCompletion status to receive IO notifications on a single >thread. That was my point, and that's what BeginRead/BeginWrite do: use WriteFile with an overlapped structure. Asynchronous IO is different then multi- threaded IO. I wasn't suggesting that overlapped IO should be avoided or that it doesn't improve the responsiveness of an application. Quite the contrary, my overlapped IO apps would be horribly unresponsive if I didn't use overlapped IO. In the general case (regardless of whether it's IO) adding more than a background thread to an GUI application does not increase performance (it may decrease performance depending on the non- .NET runtime libraries you're using--I can't comment on the .NET runtime, in this case). I do advocate avoiding doing anything but UI logic on the UI thread; but, that's usually just a case of adding a background thread and/or using overlapped IO (in my overlapped IO code, I use a background thread to perform the read/write; but, again, I'm using high-latency devices.) RAID doesn't imply that the array can handle multiple accesses at the same time; it has a queue as any other (useful) mass-storage device does. Two threads waiting for the device's queue/cache to finalize their asynchronous file access isn't going to be any more responsive than an a single background thread waiting for two asynchronous file accesses. I guess the question is, why is asynchronous IO with FileStream.BeginWrite ()/BeginRead() not good enough for your purposes? If you want to do multi- threaded access with FileStream you're either going to have to synchronize access to a single FileStream object, or use one FileStream object per thread. This is exactly how FileStream is documented; there's no bug. If you want a thread-safe object to perform multi-threaded asynchronous IO on the same file handle you'll have to write your own; that's not what FileStream is intended for. http://www.peterRitchie.com/ On Thu, 15 Dec 2005 13:29:49 -0600, John Davis <[EMAIL PROTECTED]> wrote: >See: >http://msdn.microsoft.com/library/default.asp?url=/library/en- us/dllproc/base/overlapped_str.asp >(watch the wrap on the url) > >Ritchie, > >>From: Peter Ritchie <[EMAIL PROTECTED]> >>Reply-To: "Discussion of advanced .NET topics." >><[email protected]> >>To: [email protected] >>Subject: Re: [ADVANCED-DOTNET] Possible defect in BeginWrite on FileStream >>Date: Thu, 15 Dec 2005 12:10:18 -0500 >> >>BTW, you're not supposed to use the members of OVERLAPPED (other than >>hEvent during creation). You can certainly read them, for whatever it's >>worth; but the remaining four members are documented as "Reserved for >>operating system use.". So, you can't get what you want with WriteFile >>and the OVERLAPPED struct. You're supposed to use SetFilePointer[Ex] to >>seek to a position in a file and begin writing. And, if it isn't obvious, >>there's only one file pointer to file: multi-threaded applications must >>synchronize access to the file. >> >>What do you think multi-threaded writing to a file actually buys you? >>There's only one head on the hard drive that can perform the write. Only >>one thread is really going to be writing at a time anyway; even on a multi- >>processor computer. You might be able to get two threads to write the >>cache at the same time with a multi-processor computer (you'd have to have >>at least 4 processors; the main processor is usually busy time-slicing the >>majority of operations); but, again, what does that do for you? >> >>Overlapped IO is nice because your GUI thread can be responsive while a >>potentially long read/write operation is done "on the background"; but, it >>doesn't imply multi-threaded access to a file. >> >>http://www.peterRitchie.com/ >> >> >>On Thu, 15 Dec 2005 10:01:39 -0500, John Davis <[EMAIL PROTECTED]> wrote: >> >> >Ian I'm afraid you're really mistaken on this one. >> > >> >What I'm looking for is the equivalent of a ReadFile or WriteFile call >> >where the offset is passed in the OVERLAPPED struct. The call can occur >> >through overlapped IO, or an IO completion port. The call is also >>atomic. >> > >> >There needs to be a random access async read/write of some sort which >>uses >> >the OVERLAPPED struct in the Win32 API in that it has a file offset field >> >for the read or write. If one observes the current implementation in >> >DotNetReflector, the OVERLAPPED struct is getting populated via the >> >FileStream Seek call. The OVERLAPPED struct is then used on the next >> >async write. Why implement it this way if FileStreams can only be used >>by >> >a single thread? This is a bit of a cluster if you ask me. >> > >> >On Thu, 26 May 2005 23:22:45 +0100, Ian Griffiths <[EMAIL PROTECTED] SW.CO.UK> >> >wrote: >> > >> >>Even if there were an atomic method how would that help? It wouldn't >> >>eliminate the race condition, because it would still be illegal to call >> >>the method simultaneously on multiple threads. (FileStream's >> >>documentation says it's illegal to do that.) So you'd still need to use >> >>locking or some other synchronization method to make sure your threads >> >>take it in turns even if there were an atomic method. >> >> >> >>I agree it would be cleaner to have a random access method if random >> >>access is what you're doing - a stream isn't really the right >> >>abstraction in that case. (Sadly it's the only one we're offered >> >>AFAIK.) >> >> >> >>But is this a defect? It's a bug to use a stream from multiple threads >> >>without synchronization. It would still be a bug even with the atomic >> >>method you propose. >> >> >> >> >> >>-- >> >>Ian Griffiths >> >>http://www.interact-sw.co.uk/iangblog/ >> >> >> >>> -----Original Message----- >> >>> From: John Davis >> >>> >> >>> >Are you saying that locking, Seeking, BeginWriteing, and unlocking is >> >>> not good enough? >> >>> >> >>> Well, I guess it is good enough, just doesn't seem very elegant. >> >>Would be >> >>> nice if there was an offset in there so one "atomic" call could be >> >>made. >> >>> Same as with Win32 overlapped io. >> >>> >> >>> >Does the BeginWrite not in effect take a snapshot of >> >>> the seek location when the operation starts? >> >>> >> >>> I don't know for certain if it does, I certainly hope so. Time to go >> >>> digging with reflector I guess. >> >>> >> >>> Ian Griffiths <[EMAIL PROTECTED]> wrote: >> >>> The documentation says that members of this class are not thread safe, >> >>> so multiple threads shouldn't be using this at any given instance. You >> >>> need to do some kind of locking if you have multiple threads using it. >> >>> >> >>> Are you saying that locking, Seeking, BeginWriteing, and unlocking is >> >>> not good enough? Does the BeginWrite not in effect take a snapshot of >> >>> the seek location when the operation starts? >> >>> >> >>> >> >>> > -----Original Message----- >> >>> > From: John Davis >> >>> > >> >>> > The way I understand it, the current BeginWrite on the >> >>> > FileStream object will write out it's buffer at the location of >> >>> > the last Seek. The problem comes when there are multiple threads >> >>> > using the same FileStream instance. >> >>> > >> >>> > Each one is looking to write to a different offset in the file. This >> >>> > means each thread must call FileStream.Seek followed by >> >>> > FileStream.BeginWrite Obviously, this creates a race condition. >> >>> > >> >>> > Why isn't there an async BeginWrite, on the FileStream, with an >> >>> > argument which specifies the offset within the file to write to. >> >>> > It's possible to call WriteFileEx within the Win32 API with an >> >>> > OVERLAPPED struct which specifies the offset. Why isn't this >> >>> > exposed through the framework? >> >> >> >>=================================== >> >>This list is hosted by DevelopMentor® http://www.develop.com >> >> >> >>View archives and manage your subscription(s) at >> >http://discuss.develop.com >> > >> >There needs to be a random access async read/write of some sort which >>uses >> >the OVERLAPPED struct in the Win32 API in that it has a file offset field >> >for the read or write. If one observes the current implementation in >> >DotNetReflector, the OVERLAPPED struct is getting populated via the >> >FileStream Seek call. The OVERLAPPED struct is then used on the next >> >async write. Why implement it this way if FileStreams can only be used >>by >> >a single thread? This is a bit of a cluster if you ask me. >> >>=================================== >>This list is hosted by DevelopMentor® http://www.develop.com >> >>View archives and manage your subscription(s) at http://discuss.develop.com > >=================================== >This list is hosted by DevelopMentor® http://www.develop.com > >View archives and manage your subscription(s) at http://discuss.develop.com s =================================== This list is hosted by DevelopMentor® http://www.develop.com View archives and manage your subscription(s) at http://discuss.develop.com
