No one is arguing that that wouldn't be useful to have in the runtime. But, what has that got to do with FileStream.BeginWrite()? (which works as designed and documented)
On Fri, 16 Dec 2005 15:00:37 -0500, John Davis <[EMAIL PROTECTED]> wrote: >If one uses the file offset fields in the OVERLAPPED struct and passes >this to Read/WriteFile then there is no need to call SetFilePointer, and >we have an atomic operation. As a result the race condition between >SetFilePointer and Read/WriteFile is eliminated. Similar methods should >be exposed in the framework. > >On Fri, 16 Dec 2005 09:57:10 -0500, Peter Ritchie ><[EMAIL PROTECTED]> wrote: > >>Davis, >>> >>>In OVERLAPPED there is an Offset and an OffsetHight member. These are >not >>>reserved. And your case for SetFileEx is just flat wrong. >> >>Sorry, I mis-read the internal-use-only for Offset and OffsetHigh. From >>the documentation for SetFilePointer >>http://msdn.microsoft.com/library/default.asp?url=/library/en- >>us/fileio/fs/setfilepointer.asp (http://tinyurl.com/9lpvn): >>"Be careful when you set a file pointer in a multi-threaded application. >>You must synchronize access to shared resources. For example, an >>application with threads that share a file handle, update the file >>pointer, and read from the file must protect this sequence by using a >>critical section object or mutex object." >>Sounds pretty much the same as what I was saying (admitting my mistake >>regarding the two members of OVERLAPPED). >> >>> >>>I'm guessing you haven't done much async file programming. Using IO >>>completion ports or overlapped io through this struct can dramatically >>>increase throughput for an app. This is due primarily to the fact that >>the >>>io scheduler on the hard drive (or raid array, etc.) can actually handle >>>multiple reads/writes at the same time. Having multiple outstanding io >>>requests at a time is much more efficient than only doing one at a time. >>> >> >>I've been programming IO asynchronously (not the same as multi-threaded >>IO) for years (not with disk drives, through native WriteFile/ReadFile >>with high-latency devices)... >> >>>The point is to not have your app block on a single IO operation. >Instead >>>of having 10 threads doing reads/writes one can use >WaitForMultipleObjects >>>or GetQueuedCompletion status to receive IO notifications on a single >>>thread. >> >>That was my point, and that's what BeginRead/BeginWrite do: use WriteFile >>with an overlapped structure. Asynchronous IO is different then multi- >>threaded IO. I wasn't suggesting that overlapped IO should be avoided or >>that it doesn't improve the responsiveness of an application. Quite the >>contrary, my overlapped IO apps would be horribly unresponsive if I didn't >>use overlapped IO. In the general case (regardless of whether it's IO) >>adding more than a background thread to an GUI application does not >>increase performance (it may decrease performance depending on the non- >>.NET runtime libraries you're using--I can't comment on the .NET runtime, >>in this case). I do advocate avoiding doing anything but UI logic on the >>UI thread; but, that's usually just a case of adding a background thread >>and/or using overlapped IO (in my overlapped IO code, I use a background >>thread to perform the read/write; but, again, I'm using high-latency >>devices.) >> >>RAID doesn't imply that the array can handle multiple accesses at the same >>time; it has a queue as any other (useful) mass-storage device does. Two >>threads waiting for the device's queue/cache to finalize their >>asynchronous file access isn't going to be any more responsive than an a >>single background thread waiting for two asynchronous file accesses. >> >>I guess the question is, why is asynchronous IO with FileStream.BeginWrite >>()/BeginRead() not good enough for your purposes? If you want to do >multi- >>threaded access with FileStream you're either going to have to synchronize >>access to a single FileStream object, or use one FileStream object per >>thread. This is exactly how FileStream is documented; there's no bug. >> >>If you want a thread-safe object to perform multi-threaded asynchronous IO >>on the same file handle you'll have to write your own; that's not what >>FileStream is intended for. >> >>http://www.peterRitchie.com/ >> >>On Thu, 15 Dec 2005 13:29:49 -0600, John Davis <[EMAIL PROTECTED]> wrote: >> >>>See: >>>http://msdn.microsoft.com/library/default.asp?url=/library/en- >>us/dllproc/base/overlapped_str.asp >>>(watch the wrap on the url) >>> >>>Ritchie, >> >> >>> >>>>From: Peter Ritchie <[EMAIL PROTECTED]> >>>>Reply-To: "Discussion of advanced .NET topics." >>>><[email protected]> >>>>To: [email protected] >>>>Subject: Re: [ADVANCED-DOTNET] Possible defect in BeginWrite on >>FileStream >>>>Date: Thu, 15 Dec 2005 12:10:18 -0500 >>>> >>>>BTW, you're not supposed to use the members of OVERLAPPED (other than >>>>hEvent during creation). You can certainly read them, for whatever it's >>>>worth; but the remaining four members are documented as "Reserved for >>>>operating system use.". So, you can't get what you want with WriteFile >>>>and the OVERLAPPED struct. You're supposed to use SetFilePointer[Ex] to >>>>seek to a position in a file and begin writing. And, if it isn't >>obvious, >>>>there's only one file pointer to file: multi-threaded applications must >>>>synchronize access to the file. >>>> >>>>What do you think multi-threaded writing to a file actually buys you? >>>>There's only one head on the hard drive that can perform the write. >Only >>>>one thread is really going to be writing at a time anyway; even on a >>multi- >>>>processor computer. You might be able to get two threads to write the >>>>cache at the same time with a multi-processor computer (you'd have to >>have >>>>at least 4 processors; the main processor is usually busy time-slicing >>the >>>>majority of operations); but, again, what does that do for you? >>>> >>>>Overlapped IO is nice because your GUI thread can be responsive while a >>>>potentially long read/write operation is done "on the background"; but, >>it >>>>doesn't imply multi-threaded access to a file. >>>> >>>>http://www.peterRitchie.com/ >>>> >>>> >>>>On Thu, 15 Dec 2005 10:01:39 -0500, John Davis <[EMAIL PROTECTED]> >wrote: >>>> >>>> >Ian I'm afraid you're really mistaken on this one. >>>> > >>>> >What I'm looking for is the equivalent of a ReadFile or WriteFile call >>>> >where the offset is passed in the OVERLAPPED struct. The call can >>occur >>>> >through overlapped IO, or an IO completion port. The call is also >>>>atomic. >>>> > >>>> >There needs to be a random access async read/write of some sort which >>>>uses >>>> >the OVERLAPPED struct in the Win32 API in that it has a file offset >>field >>>> >for the read or write. If one observes the current implementation in >>>> >DotNetReflector, the OVERLAPPED struct is getting populated via the >>>> >FileStream Seek call. The OVERLAPPED struct is then used on the next >>>> >async write. Why implement it this way if FileStreams can only be >used >>>>by >>>> >a single thread? This is a bit of a cluster if you ask me. >>>> > >>>> >On Thu, 26 May 2005 23:22:45 +0100, Ian Griffiths <[EMAIL PROTECTED] >>SW.CO.UK> >>>> >wrote: >>>> > >>>> >>Even if there were an atomic method how would that help? It wouldn't >>>> >>eliminate the race condition, because it would still be illegal to >>call >>>> >>the method simultaneously on multiple threads. (FileStream's >>>> >>documentation says it's illegal to do that.) So you'd still need to >>use >>>> >>locking or some other synchronization method to make sure your >threads >>>> >>take it in turns even if there were an atomic method. >>>> >> >>>> >>I agree it would be cleaner to have a random access method if random >>>> >>access is what you're doing - a stream isn't really the right >>>> >>abstraction in that case. (Sadly it's the only one we're offered >>>> >>AFAIK.) >>>> >> >>>> >>But is this a defect? It's a bug to use a stream from multiple >>threads >>>> >>without synchronization. It would still be a bug even with the >atomic >>>> >>method you propose. >>>> >> >>>> >> >>>> >>-- >>>> >>Ian Griffiths >>>> >>http://www.interact-sw.co.uk/iangblog/ >>>> >> >>>> >>> -----Original Message----- >>>> >>> From: John Davis >>>> >>> >>>> >>> >Are you saying that locking, Seeking, BeginWriteing, and unlocking >>is >>>> >>> not good enough? >>>> >>> >>>> >>> Well, I guess it is good enough, just doesn't seem very elegant. >>>> >>Would be >>>> >>> nice if there was an offset in there so one "atomic" call could be >>>> >>made. >>>> >>> Same as with Win32 overlapped io. >>>> >>> >>>> >>> >Does the BeginWrite not in effect take a snapshot of >>>> >>> the seek location when the operation starts? >>>> >>> >>>> >>> I don't know for certain if it does, I certainly hope so. Time to >>go >>>> >>> digging with reflector I guess. >>>> >>> >>>> >>> Ian Griffiths <[EMAIL PROTECTED]> wrote: >>>> >>> The documentation says that members of this class are not thread >>safe, >>>> >>> so multiple threads shouldn't be using this at any given instance. >>You >>>> >>> need to do some kind of locking if you have multiple threads using >>it. >>>> >>> >>>> >>> Are you saying that locking, Seeking, BeginWriteing, and unlocking >>is >>>> >>> not good enough? Does the BeginWrite not in effect take a snapshot >>of >>>> >>> the seek location when the operation starts? >>>> >>> >>>> >>> >>>> >>> > -----Original Message----- >>>> >>> > From: John Davis >>>> >>> > >>>> >>> > The way I understand it, the current BeginWrite on the >>>> >>> > FileStream object will write out it's buffer at the location of >>>> >>> > the last Seek. The problem comes when there are multiple threads >>>> >>> > using the same FileStream instance. >>>> >>> > >>>> >>> > Each one is looking to write to a different offset in the file. >>This >>>> >>> > means each thread must call FileStream.Seek followed by >>>> >>> > FileStream.BeginWrite Obviously, this creates a race condition. >>>> >>> > >>>> >>> > Why isn't there an async BeginWrite, on the FileStream, with an >>>> >>> > argument which specifies the offset within the file to write to. >>>> >>> > It's possible to call WriteFileEx within the Win32 API with an >>>> >>> > OVERLAPPED struct which specifies the offset. Why isn't this >>>> >>> > exposed through the framework? >>>> >> >>>> >>=================================== >>>> >>This list is hosted by DevelopMentor® http://www.develop.com >>>> >> >>>> >>View archives and manage your subscription(s) at >>>> >http://discuss.develop.com >>>> > >>>> >There needs to be a random access async read/write of some sort which >>>>uses >>>> >the OVERLAPPED struct in the Win32 API in that it has a file offset >>field >>>> >for the read or write. If one observes the current implementation in >>>> >DotNetReflector, the OVERLAPPED struct is getting populated via the >>>> >FileStream Seek call. The OVERLAPPED struct is then used on the next >>>> >async write. Why implement it this way if FileStreams can only be >used >>>>by >>>> >a single thread? This is a bit of a cluster if you ask me. >>>> >>>>=================================== >>>>This list is hosted by DevelopMentor® http://www.develop.com >>>> >>>>View archives and manage your subscription(s) at >>http://discuss.develop.com >>> >>>=================================== >>>This list is hosted by DevelopMentor® http://www.develop.com >>> >>>View archives and manage your subscription(s) at >>http://discuss.develop.com >> >>s >> >>=================================== >>This list is hosted by DevelopMentor® http://www.develop.com >> >>View archives and manage your subscription(s) at >http://discuss.develop.com > >=================================== >This list is hosted by DevelopMentor® http://www.develop.com > >View archives and manage your subscription(s) at http://discuss.develop.com =================================== This list is hosted by DevelopMentor® http://www.develop.com View archives and manage your subscription(s) at http://discuss.develop.com
