See: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/overlapped_str.asp (watch the wrap on the url)
Ritchie, In OVERLAPPED there is an Offset and an OffsetHight member. These are not reserved. And your case for SetFileEx is just flat wrong. I'm guessing you haven't done much async file programming. Using IO completion ports or overlapped io through this struct can dramatically increase throughput for an app. This is due primarily to the fact that the io scheduler on the hard drive (or raid array, etc.) can actually handle multiple reads/writes at the same time. Having multiple outstanding io requests at a time is much more efficient than only doing one at a time. The point is to not have your app block on a single IO operation. Instead of having 10 threads doing reads/writes one can use WaitForMultipleObjects or GetQueuedCompletion status to receive IO notifications on a single thread.
From: Peter Ritchie <[EMAIL PROTECTED]> Reply-To: "Discussion of advanced .NET topics." <[email protected]> To: [email protected] Subject: Re: [ADVANCED-DOTNET] Possible defect in BeginWrite on FileStream Date: Thu, 15 Dec 2005 12:10:18 -0500 BTW, you're not supposed to use the members of OVERLAPPED (other than hEvent during creation). You can certainly read them, for whatever it's worth; but the remaining four members are documented as "Reserved for operating system use.". So, you can't get what you want with WriteFile and the OVERLAPPED struct. You're supposed to use SetFilePointer[Ex] to seek to a position in a file and begin writing. And, if it isn't obvious, there's only one file pointer to file: multi-threaded applications must synchronize access to the file. What do you think multi-threaded writing to a file actually buys you? There's only one head on the hard drive that can perform the write. Only one thread is really going to be writing at a time anyway; even on a multi- processor computer. You might be able to get two threads to write the cache at the same time with a multi-processor computer (you'd have to have at least 4 processors; the main processor is usually busy time-slicing the majority of operations); but, again, what does that do for you? Overlapped IO is nice because your GUI thread can be responsive while a potentially long read/write operation is done "on the background"; but, it doesn't imply multi-threaded access to a file. http://www.peterRitchie.com/ On Thu, 15 Dec 2005 10:01:39 -0500, John Davis <[EMAIL PROTECTED]> wrote: >Ian I'm afraid you're really mistaken on this one. > >What I'm looking for is the equivalent of a ReadFile or WriteFile call >where the offset is passed in the OVERLAPPED struct. The call can occur >through overlapped IO, or an IO completion port. The call is also atomic. > >There needs to be a random access async read/write of some sort which uses >the OVERLAPPED struct in the Win32 API in that it has a file offset field >for the read or write. If one observes the current implementation in >DotNetReflector, the OVERLAPPED struct is getting populated via the >FileStream Seek call. The OVERLAPPED struct is then used on the next >async write. Why implement it this way if FileStreams can only be used by >a single thread? This is a bit of a cluster if you ask me. > >On Thu, 26 May 2005 23:22:45 +0100, Ian Griffiths <[EMAIL PROTECTED]> >wrote: > >>Even if there were an atomic method how would that help? It wouldn't >>eliminate the race condition, because it would still be illegal to call >>the method simultaneously on multiple threads. (FileStream's >>documentation says it's illegal to do that.) So you'd still need to use >>locking or some other synchronization method to make sure your threads >>take it in turns even if there were an atomic method. >> >>I agree it would be cleaner to have a random access method if random >>access is what you're doing - a stream isn't really the right >>abstraction in that case. (Sadly it's the only one we're offered >>AFAIK.) >> >>But is this a defect? It's a bug to use a stream from multiple threads >>without synchronization. It would still be a bug even with the atomic >>method you propose. >> >> >>-- >>Ian Griffiths >>http://www.interact-sw.co.uk/iangblog/ >> >>> -----Original Message----- >>> From: John Davis >>> >>> >Are you saying that locking, Seeking, BeginWriteing, and unlocking is >>> not good enough? >>> >>> Well, I guess it is good enough, just doesn't seem very elegant. >>Would be >>> nice if there was an offset in there so one "atomic" call could be >>made. >>> Same as with Win32 overlapped io. >>> >>> >Does the BeginWrite not in effect take a snapshot of >>> the seek location when the operation starts? >>> >>> I don't know for certain if it does, I certainly hope so. Time to go >>> digging with reflector I guess. >>> >>> Ian Griffiths <[EMAIL PROTECTED]> wrote: >>> The documentation says that members of this class are not thread safe, >>> so multiple threads shouldn't be using this at any given instance. You >>> need to do some kind of locking if you have multiple threads using it. >>> >>> Are you saying that locking, Seeking, BeginWriteing, and unlocking is >>> not good enough? Does the BeginWrite not in effect take a snapshot of >>> the seek location when the operation starts? >>> >>> >>> > -----Original Message----- >>> > From: John Davis >>> > >>> > The way I understand it, the current BeginWrite on the >>> > FileStream object will write out it's buffer at the location of >>> > the last Seek. The problem comes when there are multiple threads >>> > using the same FileStream instance. >>> > >>> > Each one is looking to write to a different offset in the file. This >>> > means each thread must call FileStream.Seek followed by >>> > FileStream.BeginWrite Obviously, this creates a race condition. >>> > >>> > Why isn't there an async BeginWrite, on the FileStream, with an >>> > argument which specifies the offset within the file to write to. >>> > It's possible to call WriteFileEx within the Win32 API with an >>> > OVERLAPPED struct which specifies the offset. Why isn't this >>> > exposed through the framework? >> >>=================================== >>This list is hosted by DevelopMentor® http://www.develop.com >> >>View archives and manage your subscription(s) at >http://discuss.develop.com > >There needs to be a random access async read/write of some sort which uses >the OVERLAPPED struct in the Win32 API in that it has a file offset field >for the read or write. If one observes the current implementation in >DotNetReflector, the OVERLAPPED struct is getting populated via the >FileStream Seek call. The OVERLAPPED struct is then used on the next >async write. Why implement it this way if FileStreams can only be used by >a single thread? This is a bit of a cluster if you ask me. =================================== This list is hosted by DevelopMentor® http://www.develop.com View archives and manage your subscription(s) at http://discuss.develop.com
=================================== This list is hosted by DevelopMentor® http://www.develop.com View archives and manage your subscription(s) at http://discuss.develop.com
