I appreciate your point of view as well. Disagreeing is healthy. If we all just fell in line with each other's whims and opinions we would risk heading off on a path to irrelevancy.
I also agree that I'm hard-pressed to come up with a situation where a normal use of Inline would result in concurrent compilation. Race conditions only occur when there's a race. Perhaps if a script included two or more modules that both relied upon Inline, and both hadn't previously been compiled, but that's far-fetched. On the one hand, we can reasonably assert that the CPAN testers are testing incorrectly if they're testing with concurrency, and that our failure to pass their tests is due to them not taking into consideration the fact that some modules will never be used that way. On the other hand, they have a monumental task in trying to test everything that shows up on CPAN, and expecting special case treatment for Inline might be more than they're able to accommodate. Yes, that's broken. But people look at the test results when weighing whether or not we're an appropriate solution, and unfortunately, they don't always dig into the "why" when they see a FAIL. The easiest solution for Inline would be to simply flock a semaphore file before it compiles any target code, and unlock it after it's done with the build directory. This would require only a single flock, and would prevent concurrent use of the build resources. It would have zero impact on normal users (it would never block since they would never be compiling in parallel), but would cause the tests to block for a moment occasionally. That should avoid all race conditions. Here's what I'll do: I'm going to release a dev release of Inline::CPP that does just that. If its smoke testing doesn't get any worse than it already is, I'll submit a patch for Inline that does the same thing. If you want to use it you can, and if you don't, no hard feelings. :) Dave On Sat, May 18, 2013 at 2:59 AM, <sisyph...@optusnet.com.au> wrote: > > > -----Original Message----- From: David Oswald > > >> Rob: I notice there are numerous places in the Inline distribution where >> we're opening for output but not flocking. I didn't even look at possible >> issues where we might be opening for input, reading, then opening for >> output and expecting to not tromp on another process that may have gotten >> in the middle of the cycle. > > > The key thing for me is that this absence of flocking has never (afaik) been > reported as an issue for any *user*. > Not wishing to denigrate cpan-testers (who do a fantastic job), but I've > always been loathe to address something that exists only in a smoking > set-up, for fear it's going to open up a can of worms for those who actually > want to make use of the module. > I still see the occasional cpan-tester FAIL report where 'use diagnostics;' > is a fatal error ... which only reinforces my resolve to be a little bit > cautious about the significance of the failures that the cpan-testers throw > up :-) > > >> I think that Inline probably should be >> implementing flocking on all outputs, and anytime an output follows a >> read, we should use a flocked semaphore file to maintain the lock across >> the read/write cycle. > > > I can't see that this should be necessary except perhaps for the > configuration file that gets written in the top level build directory. > I don't insist on not implementing stuff that's pointless (as along as it > comes at little cost), but I'm a bit wary of *flocking* across the board > because I can never get the damn thing to work *portably* && *silently*. > (Maybe that's just because I'm inexperienced in the use of flock.) > If we do flock, I'd rather do it only where it's proven to be necessary. > > >> I can confirm that even after fixing Inline::CPP, >> we still have race conditions that must be coming from inside of Inline, >> that rear their ugly head while executing tests in parallel. > > > Can you give me an actual "use" case - one that's outside the > cpan-tester/smoker/Test::Harness environment ? > That would probably encourage me to become a more responsible and caring > Inline maintainer :-) > > >> The fixes >> within Inline shouldn't be too difficult, but there are so many places >> that Inline interacts with files, it will be hard to assure we've gotten >> all of them. > > > My thinking is that we can't be missing too much or we would've received at > least one bug report about it from a user in the last 15 years. > > >> I suppose I could implement a semaphore flock at the >> beginning of compilation phase for Inline::CPP and release it after >> compilation phase finishes. That would make Inline::CPP concurrency-safe >> even if Inline were to remain not concurrency-safe. But that's just >> sweeping the dust under the carpet. :) > > > Well ... that'd be good enough for me. Does that work ? > > Although (as is clearly evident) I'm not all that passionate about this at > the moment, I would probably find it hard to ignore any patch that: > a) broke none of the current tests; > && > b) created no noise. > > (But I'd still prefer that we flocked only where it's demonstrably needed.) > > And I still can't help feeling that cpan-testers/Test::Harness/smokers might > be assuming something of Inline that it was never designed (or intended) to > deliver ... dunno ... again, an actual "use" case demonstrating the problem > would be helpful here. > > As you can see, David, I don't entirely agree with your POV at the moment. > But I *do* appreciate that you took the time and effort to express it. (And > I thank you for doing that !!) > > Cheers, > Rob -- David Oswald daosw...@gmail.com
