Re: [Scons-dev] Likely bug - installing side effect files
It seems one possible fix for this bug could be a warning/error if the user tries to Install() a side-effect file, and expanding/clarifying the docs for SideEffect(). Also, if Install() is permitted with a warning, the current behaviour (silently ignoring the file) must be fixed. A few other comments/questions come to mind: 1) If the contents of a side-effect file are indeterminate, how does telling SCons about this file help to have a deterministic (or convergent) build? What purpose does SideEffect() have at all? What would be the remaining valid use-cases, if Install() is not permitted? 2) The example given in docs of .pdb file is an interesting one. It looks like SideEffect() is not actually used in msvc.py. In fact, this indeterminate behaviour depends on whether the option -Fd [1] is used, what filename is specified with -Fd, and the debug information format option [2]. It is both possible and desirable, particularly in a parallel build, to either use -Z7 (foo.cc = foo.obj; no .pdb), or to use -Zi / -ZI with -Fd and a suitably unique filename (foo.cc = foo.obj, foo.pdb). However, using -Zi without explicit -Fd gives something like (foo.cc = foo.obj, vc100.pdb). (There are probably multiple bugs in the MSVC support described above...) For me, the interesting question is 1), and whether a one size fits all side-effect concept can ever be really useful in complex cases such as 2). It seems clear that for a convergent build, the builder needs to fully understand the complexities of such issues and do the right thing in all cases. The right thing may be to mark the file vc100.pdb as unstable, or just ignore it - but SideEffect() doesn't help there. While foo.pdb should be a target along with foo.obj, so it can be installed. Regards Ben [1] http://msdn.microsoft.com/en-us/library/9wst99a9.aspx [2] http://msdn.microsoft.com/en-us/library/958x11bc.aspx ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Likely bug - installing side effect files
Hi Ben, On 03.11.2014 11:48, Ben Golding wrote: It seems one possible fix for this bug could be a warning/error if the user tries to Install() a side-effect file, and expanding/clarifying the docs for SideEffect(). Also, if Install() is permitted with a warning, the current behaviour (silently ignoring the file) must be fixed. A few other comments/questions come to mind: 1) If the contents of a side-effect file are indeterminate, how does telling SCons about this file help to have a deterministic (or convergent) build? What purpose does SideEffect() have at all? What would be the remaining valid use-cases, if Install() is not permitted? I tried to explain this in my last mail. SideEffect() kind of marks a file that isn't supposed to be used as a target, but exists intermediary during the build process/step. The name of this intermediary file is always the same (it's hard-wired into the compiler/builder), so when you would run SCons in parallel mode (-j n) you could get in trouble: one build thread could get started, while another one is running...so the intermediary file (the SideEffect) could get mixed up. As far as I know, that's the scenario the SideEffect() functionality was designed for, and is actually the only situation where the method should be used. As soon as the name of the intermediary file isn't hard-wired for example, but built from the file stem of its target/source files, the usage of SideEffect() gets superfluous: each target would create a uniquely named intermediary file along with it, so there's no danger of mixing up things. Another thing that SideEffect() does, is to mark the file for getting removed on a scons -c. In the past, this has misled many writers of builders/tools, to use SideEffect() in favour of the more correct Clean()... 2) The example given in docs of .pdb file is an interesting one. It looks like SideEffect() is not actually used in msvc.py. It's very well possible that the docs aren't up-to-date, so let's do something about it! :) A better example would probably be the docbook Tool when it builds an EPUB file... In fact, this indeterminate behaviour depends on whether the option -Fd [1] is used, what filename is specified with -Fd, and the debug information format option [2]. It is both possible and desirable, particularly in a parallel build, to either use -Z7 (foo.cc = foo.obj; no .pdb), or to use -Zi / -ZI with -Fd and a suitably unique filename (foo.cc = foo.obj, foo.pdb). However, using -Zi without explicit -Fd gives something like (foo.cc = foo.obj, vc100.pdb). As I wrote above, if the filenames are guaranteed to be unique, you don't need SideEffect at all... Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Likely bug - installing side effect files
On Mon, Nov 3, 2014 at 8:44 AM, Ben Golding ben.gold...@synopsys.com wrote: Nevertheless, what effect does this marking of the side effect file actually have during the build? What can I usefully do with the object returned by SideEffect()? Does it have an effect during the parallel build? (like a mutex, restricting that only one builder instance can run concurrently if they share the same hard-coded filename) Yes, that's exactly what it does -- only one builder that produces a given side effect can run at a time. It also prevents these files from being deleted in some cases before a builder runs, at least that's what I remember. There's probably a place in Taskmaster (maybe task_prepare or something like that) that could check for a side-effect node being used as a source and emit a warning. -- Gary ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Likely bug - installing side effect files
On 03.11.2014 14:44, Ben Golding wrote: Hi Dirk, Thanks for the clarification. Nevertheless, what effect does this marking of the side effect file actually have during the build? What can I usefully do with the object returned by SideEffect()? Not very much, I'm afraid...and why would you want to anyway? I'm not a native speaker, so take it with a grain of salt please that it appears obvious to me, that declaring a file as SideEffect means that I don't really care about it. Does it have an effect during the parallel build? (like a mutex, restricting that only one builder instance can run concurrently if they share the same hard-coded filename) Yes, it does exactly that. Another thing that SideEffect() does, is to mark the file for getting removed on a scons -c. In the past, this has misled many writers of builders/tools, to use SideEffect() in favour of the more correct Clean()... If you are correct, this behaviour is certainly in contradiction with the docs: http://scons.org/doc/latest/HTML/scons-user/apd.html#f-sideeffect In the passage above I was referring to the might be removed as part of cleaning the directory in which it lives part...sorry, if this got you confused. Regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Likely bug - installing side effect files
Honestly I view sideeffect as a result ( ie a side effect) of not wanting to fix the builder up. PDB files are side effects file that might exists is some flags are set. Honestly it better to handle them as targets. Jason -Original Message- From: Scons-dev [mailto:scons-dev-boun...@scons.org] On Behalf Of Dirk Bächle Sent: Monday, November 3, 2014 8:38 AM To: scons-dev@scons.org Subject: Re: [Scons-dev] Likely bug - installing side effect files On 03.11.2014 14:44, Ben Golding wrote: Hi Dirk, Thanks for the clarification. Nevertheless, what effect does this marking of the side effect file actually have during the build? What can I usefully do with the object returned by SideEffect()? Not very much, I'm afraid...and why would you want to anyway? I'm not a native speaker, so take it with a grain of salt please that it appears obvious to me, that declaring a file as SideEffect means that I don't really care about it. Does it have an effect during the parallel build? (like a mutex, restricting that only one builder instance can run concurrently if they share the same hard-coded filename) Yes, it does exactly that. Another thing that SideEffect() does, is to mark the file for getting removed on a scons -c. In the past, this has misled many writers of builders/tools, to use SideEffect() in favour of the more correct Clean()... If you are correct, this behaviour is certainly in contradiction with the docs: http://scons.org/doc/latest/HTML/scons-user/apd.html#f-sideeffect In the passage above I was referring to the might be removed as part of cleaning the directory in which it lives part...sorry, if this got you confused. Regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
