Fair enough, we need an AsyncFileStream class which inherits from FileStream and has BeginWrite/Read methods on it which contain file offset arguments. This should be pretty easy since all the async code in there is already running off of OVERLAPPED structs.
From: Ian Griffiths <[EMAIL PROTECTED]> Reply-To: "Discussion of advanced .NET topics." <[email protected]> To: [email protected] Subject: Re: [ADVANCED-DOTNET] Possible defect in BeginWrite on FileStream Date: Sat, 17 Dec 2005 14:00:57 -0000 It seems to me that there's a clear difference between what Seek gives you and what you're asking for. (Otherwise you wouldn't be asking for it.) I admit that I chose poor terminology to describe the feature: I should have said "concurrent random access" rather than just "random access". But I stand by my assertion that it shouldn't be in Stream. The problem I have with adding this to the Stream abstraction is that all of a sudden, *every* Stream has to support it. So you undoubtedly *could* have something not dissimilar to Stream that support concurrent random access. But if you made this a feature of Stream you just made it mandatory for every implementation of Stream to support this. That makes it a non-starter for where we are today - this suggestion will never fly because Stream shipped almost 4 years ago, and there's no way to go and add this functionality retrospectively to all the Streams everyone has implemented ever since. Given where we are, the only way to add this functionality would be to introduce a new more specialized abstraction. (Either specifically to files, or possibly as some new AsyncStream abstraction.) So this leaves this discussion as an academic question: should Stream have had this functionality from the start? I think Stream's better without this, because this extra feature would increase the cost of implementing Stream. And this is the tradeoff you often get with abstractions: making them rich enough to be useful but simple enough that anyone can be bothered to implement them. Mandating that all streams that support seeking also support concurrent random access seems like a high tax to impose on implementing Stream. If everyone were clamouring for it, then it'd be reasonable. But they're not. I've worked on a wide range of .NET projects in the years since .NET shipped, and while I can appreciate in abstract that this would be a nice feature, I've never run into a scenario where this would have been useful in practice. (But I have run into scenarios where I wanted to implement Stream. So this feature would have been a net loss for me.) More tellingly, you rarely see anyone asking for it on public forums. As far as I can tell from the various mailing lists, newsgroups, and forums I've hung out on, my experience of never having come across a concrete need for this is more common than your experience. Which suggests that Microsoft made the right decision: Stream would have been the wrong place to put support for concurrent random access. So even if you were starting from a clean slate, and wanted v1.0 of the framework to offer this functionality (and I agree it would be nice to have, despite the fact that I've never actually needed it) I still think it would be a mistake to put something this specialized into something as generic as Stream. It's at best a poor match (and at worst, inappropriate) for most implementations of Stream. For the implementations of Stream for which it's appropriate, it's not useful most of the time. To steal a phrase that Ron Jeffries recently used on the TDD mailing list: " it would be a bit like having a #10 Torx driver as part of my hammer. Occasionally useful but also an odd addition." Of course it's hugely frustrating if you happen to hit the case where you want to do something that you know would be pretty easy for the underlying implementation you happen to be using. But that doesn't translate into an argument that Stream is designed wrong. -- Ian Griffiths http://www.interact-sw.co.uk/iangblog/ > -----Original Message----- > From: John Davis > > Yes, I disagree with your assertion stating the Stream interface is the > wrong abstraction for random access because it already contains a Seek > method. The random access is already there. > > So, for streams where (CanSeek == true) there should also be a > BeginRead/Write(file_offset, ...) which is threadsafe. > > Yes, you're right the docs say it isn't threadsafe. But for heaven sake, > surely we can add a class that is threadsafe for file io in the same way > the > Win32 API provides. How are people supposed to write high throughput > server > software without a good async managed file io class? Apparently everyone > is > still using C++. > > As you may have guessed, I've already hit this on my current project. > Lots > of locks to compensate for the above oversight. > > >From: Ian Griffiths <[EMAIL PROTECTED]> > >Reply-To: "Discussion of advanced .NET topics." > ><[email protected]> > >To: [email protected] > >Subject: Re: [ADVANCED-DOTNET] Possible defect in BeginWrite on > FileStream > >Date: Thu, 15 Dec 2005 22:42:36 -0000 > > > >John Davis wrote: > > > Ian I'm afraid you're really mistaken on this one. > > > >Which bit do you disagree with? > > > >Do you disagree that Stream is the wrong abstraction for random access? > > > >Do you disagree that if a class is documented as not safe for > >multi-threaded access, then it doesn't really matter what new methods > >you add, you still can't use it from multiple threads? > > > >Do you disagree with the claim that Stream offering stream-like > >behaviour is a design limitation and not an actual defect? (The absence > >of a random access API might be regarded as a defect, but that doesn't > >mean there's necessarily a problem with the stream API - it does what > >it's designed to do.) > > > >You've obviously thought long and hard about it - almost 7 months > >according to the email timestamps. :) But I can't tell what I said that > >you think I'm "really mistaken on". > > > >I'm not disagreeing that random access (as opposed to stream-like > >access) would be nice to have. (Although it's not something I happen to > >have run into a need for in any .NET projects yet.) I'm just saying > >Stream's the wrong place for it. > > > >In particular, bear in mind that not all Streams are implemented on top > >of something that necessarily has intrinsic OVERLAPPED-like > >functionality. (Not all streams are files or file-like handles.) > > > > > > > > > What I'm looking for is the equivalent of a ReadFile or WriteFile > > > call here the offset is passed in the OVERLAPPED struct. > > > >I agree there are circumstances in which this would be handy. But I > >stand by my assertions that Stream is the wrong abstraction for this. > >Not every Stream is a file. Stream is not meant to be an abstract > >representation of a file - it's more widely applicable than that. (Hence > >its narrower API - the more specific an API an abstraction has, the less > >broadly applicable it is.) > > > > > > > The call can occur through overlapped IO, or an IO completion > > > port. The call is also atomic. > > > >And it's not presenting a stream-like abstraction. I think that's at > >the heart of the issue. > > > > > > > > > There needs to be a random access async read/write of some > > > sort which uses the OVERLAPPED struct in the Win32 API in that > > > it has a file offset field for the read or write. > > > >Sure, just probably not Stream. > > > >I guess if you changed nature of the Stream API by adding new > >non-streamish functionality and also changed the classes so they > >permitted concurrent access then that'd solve the problem. (Except for > >the fact that this might break a lot of existing streams because they > >don't implement this feature right now...) > > > >But aren't you introducing a whole new abstraction? > > > > > > > If one observes the current implementation in > > > DotNetReflector, the OVERLAPPED struct is getting populated > > > via the FileStream Seek call. The OVERLAPPED struct is then > > > used on the next async write. Why implement it this way if > > > FileStreams can only be used by a single thread? > > > >Err...isn't that a distinctly sequential approach to the problem? It > >presumes that the call to Seek and the async write will follow on one > >after the other without interference. > > > >You'd need a completely different implementation if you wanted to > >support non-sequential use wouldn't you? > > > > > >-- > >Ian Griffiths > > > > > >On Thu, 26 May 2005 23:22:45 +0100, Ian Griffiths > ><[EMAIL PROTECTED]> > >wrote: > > > > >Even if there were an atomic method how would that help? It wouldn't > > >eliminate the race condition, because it would still be illegal to call > > >the method simultaneously on multiple threads. (FileStream's > > >documentation says it's illegal to do that.) So you'd still need to > >use > > >locking or some other synchronization method to make sure your threads > > >take it in turns even if there were an atomic method. > > > > > >I agree it would be cleaner to have a random access method if random > > >access is what you're doing - a stream isn't really the right > > >abstraction in that case. (Sadly it's the only one we're offered > > >AFAIK.) > > > > > >But is this a defect? It's a bug to use a stream from multiple threads > > >without synchronization. It would still be a bug even with the atomic > > >method you propose. > > > > > > > > >-- > > >Ian Griffiths > > >http://www.interact-sw.co.uk/iangblog/ > > > > > >> -----Original Message----- > > >> From: John Davis > > >> > > >> >Are you saying that locking, Seeking, BeginWriteing, and unlocking > >is > > >> not good enough? > > >> > > >> Well, I guess it is good enough, just doesn't seem very elegant. > > >Would be > > >> nice if there was an offset in there so one "atomic" call could be > > >made. > > >> Same as with Win32 overlapped io. > > >> > > >> >Does the BeginWrite not in effect take a snapshot of > > >> the seek location when the operation starts? > > >> > > >> I don't know for certain if it does, I certainly hope so. Time to go > > >> digging with reflector I guess. > > >> > > >> Ian Griffiths <[EMAIL PROTECTED]> wrote: > > >> The documentation says that members of this class are not thread > >safe, > > >> so multiple threads shouldn't be using this at any given instance. > >You > > >> need to do some kind of locking if you have multiple threads using > >it. > > >> > > >> Are you saying that locking, Seeking, BeginWriteing, and unlocking is > > >> not good enough? Does the BeginWrite not in effect take a snapshot of > > >> the seek location when the operation starts? > > > >=================================== > >This list is hosted by DevelopMentor(r) http://www.develop.com > > > >View archives and manage your subscription(s) at > http://discuss.develop.com > > =================================== > This list is hosted by DevelopMentor(r) http://www.develop.com > > View archives and manage your subscription(s) at > http://discuss.develop.com =================================== This list is hosted by DevelopMentor® http://www.develop.com View archives and manage your subscription(s) at http://discuss.develop.com
=================================== This list is hosted by DevelopMentor® http://www.develop.com View archives and manage your subscription(s) at http://discuss.develop.com
