On Mon, 2008-08-25 at 23:09 +0800, Adrian Chadd wrote: > Um, where/when was all of this discussed and decided? I recall seeing > the discussion where it was pointed out something was needed but I > don't recall anything being decided..
I do not think there is a single place or time where "all of this" was discussed or decided, but most of my responses are simply documenting _existing_ Squid3 code. The buffering issues (which, again, are a separate topic, IMO) are still being discussed. I have already referred to Amos' BetterStringBuffer wiki page. Now it looks like there is a parallel effort: Kinkie started to attack the same problem today with a "out-of-tree merge-friendly prototype". I hope we can merge the two efforts and I did ask the questions to identify the differences among BetterStringBuffer and prototype approaches. HTH, Alex. > 2008/8/25 Alex Rousskov <[EMAIL PROTECTED]>: > > > > On Mon, 2008-08-25 at 12:26 +0800, Adrian Chadd wrote: > >> There's no way to guarantee sync termination of comm IO events in some > >> environments, eg aio_read and aio_write on kernel-AIO platforms like > >> FreeBSD. > > > > Yes, the Squid3 design I am documenting guarantees call[back] > > cancellation only. The underlying I/O may have already happened, is > > happening, or may happen despite the call cancellation. > > > >> You could hide that behaviour from the upper layers by making > >> comm_close() not fully complete until IO completes (which it does for > >> SSL, iirc); but any user resources passed into the comm call (such as > >> a buffer) have to be reference counted before you pull that off as the > >> kernel may still end up doing something with the buffer past the > >> users' cancellation request. > > > > Preserving buffers and other resources is the responsibility of the I/O > > registration object that survives until nobody cares about that I/O. > > Different I/O schemes may need to implement the I/O registration API > > differently. I agree that I/O buffers should be eventually refcounted to > > make the implementations simpler and cleaner. > > > >> I agree that the comm_close() use as a "clean me up" trigger is not a > >> good idea; comm_close() should just be a method of closing the > >> filedescriptor and -comm- resources associated with it. > >> > >> There's still quite a bit of work though in making the existing code > >> "do" what either/both of us are suggesting. Has there actually been a > >> discussion and a plan for what this should look like long-term? > > > > The distance between the current Squid3 code and the design I am > > documenting is not huge anymore and we are working on shrinking it > > further. And yes, we have discussed Squid modules, AsyncCalls, Comm, and > > buffering quite a few times already. I think there was consensus on > > these issues except for, perhaps, buffer handling which needs more > > discussions (Amos has a corresponding Feature page on the wiki). > > > > I have not seen Amos slides from the Australia meetup yet but I hope > > they do not suggest any radical changes compared to what has been > > discussed before that meeting. > > > > Cheers, > > > > Alex. > > > > > >> 2008/8/25 Alex Rousskov <[EMAIL PROTECTED]>: > >> > On Sat, 2008-08-23 at 08:31 +0800, Adrian Chadd wrote: > >> >> I'm also probably going to go for > >> >> "will always complete" versus the current cancellation models in > >> >> Squid-3. Ie, a cancelled IO transaction will still call the completion > >> >> callback with some kind of "error/cancelled" status; the callback > >> >> function can then cleanup as appropriate. > >> > > >> > I doubt it is a good idea to try to force every job ordering the I/O to > >> > wait for the I/O completion call. If the job has to terminate for some > >> > reason, it should cancel its Comm call(s) and focus on the termination > >> > business. > >> > > >> > Allowing the job to cancel its interest in I/O does not complicate the > >> > Comm code (that does not care what the job does with the call). It does > >> > simplify the job code (that does not have to "idle" while waiting for a > >> > Comm call it no longer cares about). > >> > > >> > What often needs a cleanup is job code that uses comm_close() to > >> > indirectly trigger that job cleanup and destruction. In most cases, > >> > there are better ways to clean and terminate the job than relying on > >> > Comm to eventually initiate that process even though the termination > >> > reason had nothing to do with the pending Comm I/Os. > >> > > >> > HTH, > >> > > >> > Alex. > >> > > >> > > >> > > > > >
