Re: [Scons-dev] Likely bug - installing side effect files
It seems one possible fix for this bug could be a warning/error if the user tries to Install() a side-effect file, and expanding/clarifying the docs for SideEffect(). Also, if Install() is permitted with a warning, the current behaviour (silently ignoring the file) must be fixed. A few other comments/questions come to mind: 1) If the contents of a side-effect file are indeterminate, how does telling SCons about this file help to have a deterministic (or convergent) build? What purpose does SideEffect() have at all? What would be the remaining valid use-cases, if Install() is not permitted? 2) The example given in docs of .pdb file is an interesting one. It looks like SideEffect() is not actually used in msvc.py. In fact, this indeterminate behaviour depends on whether the option -Fd [1] is used, what filename is specified with -Fd, and the debug information format option [2]. It is both possible and desirable, particularly in a parallel build, to either use -Z7 (foo.cc = foo.obj; no .pdb), or to use -Zi / -ZI with -Fd and a suitably unique filename (foo.cc = foo.obj, foo.pdb). However, using -Zi without explicit -Fd gives something like (foo.cc = foo.obj, vc100.pdb). (There are probably multiple bugs in the MSVC support described above...) For me, the interesting question is 1), and whether a one size fits all side-effect concept can ever be really useful in complex cases such as 2). It seems clear that for a convergent build, the builder needs to fully understand the complexities of such issues and do the right thing in all cases. The right thing may be to mark the file vc100.pdb as unstable, or just ignore it - but SideEffect() doesn't help there. While foo.pdb should be a target along with foo.obj, so it can be installed. Regards Ben [1] http://msdn.microsoft.com/en-us/library/9wst99a9.aspx [2] http://msdn.microsoft.com/en-us/library/958x11bc.aspx ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Likely bug - installing side effect files
Hi Ben, On 03.11.2014 11:48, Ben Golding wrote: It seems one possible fix for this bug could be a warning/error if the user tries to Install() a side-effect file, and expanding/clarifying the docs for SideEffect(). Also, if Install() is permitted with a warning, the current behaviour (silently ignoring the file) must be fixed. A few other comments/questions come to mind: 1) If the contents of a side-effect file are indeterminate, how does telling SCons about this file help to have a deterministic (or convergent) build? What purpose does SideEffect() have at all? What would be the remaining valid use-cases, if Install() is not permitted? I tried to explain this in my last mail. SideEffect() kind of marks a file that isn't supposed to be used as a target, but exists intermediary during the build process/step. The name of this intermediary file is always the same (it's hard-wired into the compiler/builder), so when you would run SCons in parallel mode (-j n) you could get in trouble: one build thread could get started, while another one is running...so the intermediary file (the SideEffect) could get mixed up. As far as I know, that's the scenario the SideEffect() functionality was designed for, and is actually the only situation where the method should be used. As soon as the name of the intermediary file isn't hard-wired for example, but built from the file stem of its target/source files, the usage of SideEffect() gets superfluous: each target would create a uniquely named intermediary file along with it, so there's no danger of mixing up things. Another thing that SideEffect() does, is to mark the file for getting removed on a scons -c. In the past, this has misled many writers of builders/tools, to use SideEffect() in favour of the more correct Clean()... 2) The example given in docs of .pdb file is an interesting one. It looks like SideEffect() is not actually used in msvc.py. It's very well possible that the docs aren't up-to-date, so let's do something about it! :) A better example would probably be the docbook Tool when it builds an EPUB file... In fact, this indeterminate behaviour depends on whether the option -Fd [1] is used, what filename is specified with -Fd, and the debug information format option [2]. It is both possible and desirable, particularly in a parallel build, to either use -Z7 (foo.cc = foo.obj; no .pdb), or to use -Zi / -ZI with -Fd and a suitably unique filename (foo.cc = foo.obj, foo.pdb). However, using -Zi without explicit -Fd gives something like (foo.cc = foo.obj, vc100.pdb). As I wrote above, if the filenames are guaranteed to be unique, you don't need SideEffect at all... Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Likely bug - installing side effect files
On Mon, Nov 3, 2014 at 8:44 AM, Ben Golding ben.gold...@synopsys.com wrote: Nevertheless, what effect does this marking of the side effect file actually have during the build? What can I usefully do with the object returned by SideEffect()? Does it have an effect during the parallel build? (like a mutex, restricting that only one builder instance can run concurrently if they share the same hard-coded filename) Yes, that's exactly what it does -- only one builder that produces a given side effect can run at a time. It also prevents these files from being deleted in some cases before a builder runs, at least that's what I remember. There's probably a place in Taskmaster (maybe task_prepare or something like that) that could check for a side-effect node being used as a source and emit a warning. -- Gary ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Likely bug - installing side effect files
On 03.11.2014 14:44, Ben Golding wrote: Hi Dirk, Thanks for the clarification. Nevertheless, what effect does this marking of the side effect file actually have during the build? What can I usefully do with the object returned by SideEffect()? Not very much, I'm afraid...and why would you want to anyway? I'm not a native speaker, so take it with a grain of salt please that it appears obvious to me, that declaring a file as SideEffect means that I don't really care about it. Does it have an effect during the parallel build? (like a mutex, restricting that only one builder instance can run concurrently if they share the same hard-coded filename) Yes, it does exactly that. Another thing that SideEffect() does, is to mark the file for getting removed on a scons -c. In the past, this has misled many writers of builders/tools, to use SideEffect() in favour of the more correct Clean()... If you are correct, this behaviour is certainly in contradiction with the docs: http://scons.org/doc/latest/HTML/scons-user/apd.html#f-sideeffect In the passage above I was referring to the might be removed as part of cleaning the directory in which it lives part...sorry, if this got you confused. Regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Likely bug - installing side effect files
Honestly I view sideeffect as a result ( ie a side effect) of not wanting to fix the builder up. PDB files are side effects file that might exists is some flags are set. Honestly it better to handle them as targets. Jason -Original Message- From: Scons-dev [mailto:scons-dev-boun...@scons.org] On Behalf Of Dirk Bächle Sent: Monday, November 3, 2014 8:38 AM To: scons-dev@scons.org Subject: Re: [Scons-dev] Likely bug - installing side effect files On 03.11.2014 14:44, Ben Golding wrote: Hi Dirk, Thanks for the clarification. Nevertheless, what effect does this marking of the side effect file actually have during the build? What can I usefully do with the object returned by SideEffect()? Not very much, I'm afraid...and why would you want to anyway? I'm not a native speaker, so take it with a grain of salt please that it appears obvious to me, that declaring a file as SideEffect means that I don't really care about it. Does it have an effect during the parallel build? (like a mutex, restricting that only one builder instance can run concurrently if they share the same hard-coded filename) Yes, it does exactly that. Another thing that SideEffect() does, is to mark the file for getting removed on a scons -c. In the past, this has misled many writers of builders/tools, to use SideEffect() in favour of the more correct Clean()... If you are correct, this behaviour is certainly in contradiction with the docs: http://scons.org/doc/latest/HTML/scons-user/apd.html#f-sideeffect In the passage above I was referring to the might be removed as part of cleaning the directory in which it lives part...sorry, if this got you confused. Regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Likely bug - installing side effect files
Gary, On 01.11.2014 12:03, Gary Oberbrunner wrote: SideEffect may be the root cause of the problem, but Install should not behave this way, IMHO. If you pass some nodes to Install, it should either install them or fail the build with an error (don't know how to build... or similar). It shouldn't ever _not_ install them silently. I gave this some more thought, and when we talk about proper targets, or source files that always exist, I agree with you. Install should just work... However, a file marked as SideEffect is a different story... And that's because it has a fixed name, but can have different contents... depending on whether you're currently building a.foo or b.foo, as introduced in my last example. This means that if we supported it as full target, we would soon run into trouble with things like: scons a.foo b.foo install vs. scons b.foo a.foo install . We're primarily a build system, not an install system, and care about secure and stable builds...which also should include convergence at the top of our priority list. With this I mean, that a user expects that the build converges to a settled state (I do, at least), where all targets are properly updated...independent of the order in which the single build steps for the targets are executed. And it's obvious to me that we'd start violating this principle, when treating SideEffects as fully equivalent targets. Arguing against myself :-), we could change the doc for SideEffect to indicate that nodes that are SideEffects are not guaranteed to be created, and may or may not be ignored by targets that use them as sources. But, this kind of indeterminacy doesn't seem like a good user experience to me. The problem I described above, doesn't yield a good user experience either... ;) In my opinion, this is the point where we have to educate (horrible word, but I can't think of a better term right now) the user to not use SideEffect as a workaround for properly defining a Builder/Emitter instead. SideEffect should be reserved for files that only the compiler (or any other build step) cares about. If the *user* cares about a target file, and wants to work with it in the DAG---by using it as input file to another build step for example---he has to define it as a proper target (=Emitter). On Fri, Oct 31, 2014 at 7:42 PM, William Blevins wblevins...@gmail.com wrote: Team, Not to be contrary here, but I think personal opinions should be postponed until we determine if the definition of SideEffect per the SCons User Guide matches the actual behavior. http://www.scons.org/doc/production/HTML/scons-user.html [...] Reading the description, I think that the SideEffect behavior doesn't cover the Depends behaviors which Andrew desires. I agree, in the sense that the usage of SideEffect is simply the wrong approach for this use case. From my angle, the documentation matches the current behaviour just fine... However, it's probably a good idea to make it more clear that defining a SideEffect file means that the user doesn't care about what happens with the file. So, he can't rely on the file to be properly updated, or have a determined content at a given time. Therefore, he also can't Install/Copy/OtherBuilder() the file...because its contents are actually unknown. So much for my latest thoughts about this, I hope they make some sense. Regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Likely bug - installing side effect files
Well said Dirk! +1 On Sun, Nov 2, 2014 at 4:25 AM, Dirk Bächle tshor...@gmx.de wrote: Gary, On 01.11.2014 12:03, Gary Oberbrunner wrote: SideEffect may be the root cause of the problem, but Install should not behave this way, IMHO. If you pass some nodes to Install, it should either install them or fail the build with an error (don't know how to build... or similar). It shouldn't ever _not_ install them silently. I gave this some more thought, and when we talk about proper targets, or source files that always exist, I agree with you. Install should just work... However, a file marked as SideEffect is a different story... And that's because it has a fixed name, but can have different contents... depending on whether you're currently building a.foo or b.foo, as introduced in my last example. This means that if we supported it as full target, we would soon run into trouble with things like: scons a.foo b.foo install vs. scons b.foo a.foo install . We're primarily a build system, not an install system, and care about secure and stable builds...which also should include convergence at the top of our priority list. With this I mean, that a user expects that the build converges to a settled state (I do, at least), where all targets are properly updated...independent of the order in which the single build steps for the targets are executed. And it's obvious to me that we'd start violating this principle, when treating SideEffects as fully equivalent targets. Arguing against myself :-), we could change the doc for SideEffect to indicate that nodes that are SideEffects are not guaranteed to be created, and may or may not be ignored by targets that use them as sources. But, this kind of indeterminacy doesn't seem like a good user experience to me. The problem I described above, doesn't yield a good user experience either... ;) In my opinion, this is the point where we have to educate (horrible word, but I can't think of a better term right now) the user to not use SideEffect as a workaround for properly defining a Builder/Emitter instead. SideEffect should be reserved for files that only the compiler (or any other build step) cares about. If the *user* cares about a target file, and wants to work with it in the DAG---by using it as input file to another build step for example---he has to define it as a proper target (=Emitter). On Fri, Oct 31, 2014 at 7:42 PM, William Blevins wblevins...@gmail.com wrote: Team, Not to be contrary here, but I think personal opinions should be postponed until we determine if the definition of SideEffect per the SCons User Guide matches the actual behavior. http://www.scons.org/doc/production/HTML/scons-user.html [...] Reading the description, I think that the SideEffect behavior doesn't cover the Depends behaviors which Andrew desires. I agree, in the sense that the usage of SideEffect is simply the wrong approach for this use case. From my angle, the documentation matches the current behaviour just fine... However, it's probably a good idea to make it more clear that defining a SideEffect file means that the user doesn't care about what happens with the file. So, he can't rely on the file to be properly updated, or have a determined content at a given time. Therefore, he also can't Install/Copy/OtherBuilder() the file...because its contents are actually unknown. So much for my latest thoughts about this, I hope they make some sense. Regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Likely bug - installing side effect files
I agree with Dirk here. If multiple actions can have the same side-effect files, then Install cannot have a correct behavior. The behavior will always be ambiguous. If the files are always unique to the build actions, then they are not side-effects at all and should be correctly added to an emitter. V/R, William On Sun, Nov 2, 2014 at 2:30 PM, Bill Deegan b...@baddogconsulting.com wrote: Well said Dirk! +1 On Sun, Nov 2, 2014 at 4:25 AM, Dirk Bächle tshor...@gmx.de wrote: Gary, On 01.11.2014 12:03, Gary Oberbrunner wrote: SideEffect may be the root cause of the problem, but Install should not behave this way, IMHO. If you pass some nodes to Install, it should either install them or fail the build with an error (don't know how to build... or similar). It shouldn't ever _not_ install them silently. I gave this some more thought, and when we talk about proper targets, or source files that always exist, I agree with you. Install should just work... However, a file marked as SideEffect is a different story... And that's because it has a fixed name, but can have different contents... depending on whether you're currently building a.foo or b.foo, as introduced in my last example. This means that if we supported it as full target, we would soon run into trouble with things like: scons a.foo b.foo install vs. scons b.foo a.foo install . We're primarily a build system, not an install system, and care about secure and stable builds...which also should include convergence at the top of our priority list. With this I mean, that a user expects that the build converges to a settled state (I do, at least), where all targets are properly updated...independent of the order in which the single build steps for the targets are executed. And it's obvious to me that we'd start violating this principle, when treating SideEffects as fully equivalent targets. Arguing against myself :-), we could change the doc for SideEffect to indicate that nodes that are SideEffects are not guaranteed to be created, and may or may not be ignored by targets that use them as sources. But, this kind of indeterminacy doesn't seem like a good user experience to me. The problem I described above, doesn't yield a good user experience either... ;) In my opinion, this is the point where we have to educate (horrible word, but I can't think of a better term right now) the user to not use SideEffect as a workaround for properly defining a Builder/Emitter instead. SideEffect should be reserved for files that only the compiler (or any other build step) cares about. If the *user* cares about a target file, and wants to work with it in the DAG---by using it as input file to another build step for example---he has to define it as a proper target (=Emitter). On Fri, Oct 31, 2014 at 7:42 PM, William Blevins wblevins...@gmail.com wrote: Team, Not to be contrary here, but I think personal opinions should be postponed until we determine if the definition of SideEffect per the SCons User Guide matches the actual behavior. http://www.scons.org/doc/production/HTML/scons-user.html [...] Reading the description, I think that the SideEffect behavior doesn't cover the Depends behaviors which Andrew desires. I agree, in the sense that the usage of SideEffect is simply the wrong approach for this use case. From my angle, the documentation matches the current behaviour just fine... However, it's probably a good idea to make it more clear that defining a SideEffect file means that the user doesn't care about what happens with the file. So, he can't rely on the file to be properly updated, or have a determined content at a given time. Therefore, he also can't Install/Copy/OtherBuilder() the file...because its contents are actually unknown. So much for my latest thoughts about this, I hope they make some sense. Regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Likely bug - installing side effect files
I'd almost say we should through an exception when a sideffect file is
specified as a source (for any builder)
-Bill
On Sat, Nov 1, 2014 at 4:03 AM, Gary Oberbrunner ga...@oberbrunner.com
wrote:
SideEffect may be the root cause of the problem, but Install should
not behave this way, IMHO. If you pass some nodes to Install, it
should either install them or fail the build with an error (don't
know how to build... or similar). It shouldn't ever _not_ install
them silently.
Arguing against myself :-), we could change the doc for SideEffect to
indicate that nodes that are SideEffects are not guaranteed to be
created, and may or may not be ignored by targets that use them as
sources. But, this kind of indeterminacy doesn't seem like a good
user experience to me.
On Fri, Oct 31, 2014 at 7:42 PM, William Blevins wblevins...@gmail.com
wrote:
Team,
Not to be contrary here, but I think personal opinions should be
postponed
until we determine if the definition of SideEffect per the SCons User
Guide
matches the actual behavior.
http://www.scons.org/doc/production/HTML/scons-user.html
SideEffect(side_effect, target) , env.SideEffect(side_effect, target)
Declares side_effect as a side effect of building target. Both
side_effect
and target can be a list, a file name, or a node. A side effect is a
target
file that is created or updated as a side effect of building other
targets.
For example, a Windows PDB file is created as a side effect of building
the
.obj files for a static library, and various log files are created
updated
as side effects of various TeX commands. If a target is a side effect of
multiple build commands, scons will ensure that only one set of
commands is
executed at a time. Consequently, you only need to use this method for
side-effect targets that are built as a result of multiple build
commands.
Because multiple build commands may update the same side effect file, by
default the side_effect target is not automatically removed when the
target
is removed by the -c option. (Note, however, that the side_effect might
be
removed as part of cleaning the directory in which it lives.) If you
want to
make sure the side_effect is cleaned whenever a specific target is
cleaned,
you must specify this explicitly with the Clean or env.Clean function.
Reading the description, I think that the SideEffect behavior doesn't
cover
the Depends behaviors which Andrew desires.
V/R,
William
On Fri, Oct 31, 2014 at 7:22 PM, Bill Deegan b...@baddogconsulting.com
wrote:
SideEffect() by definition (to me at least) should be informing SCons
that
a file is produced, but it is not important. Allowing SCons to be
aware of
it.
It seems like using that file as an input in any part of your build
would
violate such an idea.
So I think I'm agreeing with Dirk on this one.
Regardless of however the code might be defined today. I think it's
important to be able to tell SCons about such a type of file.
-Bill
On Fri, Oct 31, 2014 at 12:20 PM, Dirk Bächle tshor...@gmx.de wrote:
On 31.10.2014 20:04, Gary Oberbrunner wrote:
On Fri, Oct 31, 2014 at 2:55 PM, Dirk Bächle tshor...@gmx.de wrote:
I don't think there is anything to fix here...and it's no bug for me
either.
Please read the man page for the definition
of a SideEffect, and when it should be used. My understanding is,
that
in
your case the conditions do not apply...so you should rather define
an
Emitter (see http://www.scons.org/wiki/ToolsForFools for example)
that
adds
sf0 and sf1 to the list of targets.
Then the Install() should work as expected.
I disagree. He's explicitly passing the files in sf to Install();
Install() should always (try to) install all the files given as its
sources. Whether they're created as side effects or anything else
_shouldn't_ matter.
Hmm, do you mean all files or all targets? I really don't regard
SideEffects to be full targets...but maybe I'm overlooking something.
I'm
just thinking about the case where I use a foo compiler. It always
creates
a file named.fix, containing some internal info that's input for the
build
step itself.
So, for scheduling single builds with this compiler correctly, in
concurrency speak: locking it with a kind of monitor for parallel
mode, I
define the file named.fix as SideEffect in my foo Builder.
If I then say something like:
env = Environment(tools=['foo'])
a = env.Foo('a.foo','a.in')
b = env.Foo('b.foo','b.in')
env.Install('/install', [a, b, 'named.fix'])
which version of named.fix gets installed? Which version *should* get
installed? And is the intended ordering guaranteed for any invocation
of -j
n?
You're probably right, but the above is what's rolling through my head
right now...all kinds of clarifications and comments are welcome. ;)
Dirk
Re: [Scons-dev] Likely bug - installing side effect files
Hi there, On 30.10.2014 16:01, Ben Golding wrote: Thanks Gary. http://scons.tigris.org/issues/show_bug.cgi?id=2982 I started to add some additional trace to Taskmaster.py, and compare the logs from sequential vs. parallel builds. The bug seems to relate to how the node states are used for the side-effect nodes: - In executed_with_callbacks(), the side-effect state is set to NODE_NO_STATE. - In postprocess(), the logic to decrement the parent ref count is only active for a side-effect node with state NODE_EXECUTING. Although this helped me to understand the bug a bit better, I'm not sure what is the correct fix. I don't think there is anything to fix here...and it's no bug for me either. Please read the man page for the definition of a SideEffect, and when it should be used. My understanding is, that in your case the conditions do not apply...so you should rather define an Emitter (see http://www.scons.org/wiki/ToolsForFools for example) that adds sf0 and sf1 to the list of targets. Then the Install() should work as expected. Just my 2 cents. Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Likely bug - installing side effect files
On Fri, Oct 31, 2014 at 2:55 PM, Dirk Bächle tshor...@gmx.de wrote: I don't think there is anything to fix here...and it's no bug for me either. Please read the man page for the definition of a SideEffect, and when it should be used. My understanding is, that in your case the conditions do not apply...so you should rather define an Emitter (see http://www.scons.org/wiki/ToolsForFools for example) that adds sf0 and sf1 to the list of targets. Then the Install() should work as expected. I disagree. He's explicitly passing the files in sf to Install(); Install() should always (try to) install all the files given as its sources. Whether they're created as side effects or anything else _shouldn't_ matter. -- Gary ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Likely bug - installing side effect files
SideEffect() by definition (to me at least) should be informing SCons that
a file is produced, but it is not important. Allowing SCons to be aware of
it.
It seems like using that file as an input in any part of your build would
violate such an idea.
So I think I'm agreeing with Dirk on this one.
Regardless of however the code might be defined today. I think it's
important to be able to tell SCons about such a type of file.
-Bill
On Fri, Oct 31, 2014 at 12:20 PM, Dirk Bächle tshor...@gmx.de wrote:
On 31.10.2014 20:04, Gary Oberbrunner wrote:
On Fri, Oct 31, 2014 at 2:55 PM, Dirk Bächle tshor...@gmx.de wrote:
I don't think there is anything to fix here...and it's no bug for me
either.
Please read the man page for the definition
of a SideEffect, and when it should be used. My understanding is, that in
your case the conditions do not apply...so you should rather define an
Emitter (see http://www.scons.org/wiki/ToolsForFools for example) that
adds
sf0 and sf1 to the list of targets.
Then the Install() should work as expected.
I disagree. He's explicitly passing the files in sf to Install();
Install() should always (try to) install all the files given as its
sources. Whether they're created as side effects or anything else
_shouldn't_ matter.
Hmm, do you mean all files or all targets? I really don't regard
SideEffects to be full targets...but maybe I'm overlooking something. I'm
just thinking about the case where I use a foo compiler. It always
creates a file named.fix, containing some internal info that's input for
the build step itself.
So, for scheduling single builds with this compiler correctly, in
concurrency speak: locking it with a kind of monitor for parallel mode, I
define the file named.fix as SideEffect in my foo Builder.
If I then say something like:
env = Environment(tools=['foo'])
a = env.Foo('a.foo','a.in')
b = env.Foo('b.foo','b.in')
env.Install('/install', [a, b, 'named.fix'])
which version of named.fix gets installed? Which version *should* get
installed? And is the intended ordering guaranteed for any invocation of
-j n?
You're probably right, but the above is what's rolling through my head
right now...all kinds of clarifications and comments are welcome. ;)
Dirk
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Likely bug - installing side effect files
Team,
Not to be contrary here, but I think personal opinions should be postponed
until we determine if the definition of SideEffect per the SCons User Guide
matches the actual behavior.
http://www.scons.org/doc/production/HTML/scons-user.html
SideEffect(side_effect, target) , env.SideEffect(side_effect, target)
Declares side_effect as a side effect of building target. Both side_effect
and target can be a list, a file name, or a node. A side effect is a
target file that is created or updated as a side effect of building other
targets. For example, a Windows PDB file is created as a side effect of
building the .obj files for a static library, and various log files are
created updated as side effects of various TeX commands. If a target is a
side effect of multiple build commands, scons will ensure that only one
set of commands is executed at a time. Consequently, you only need to use
this method for side-effect targets that are built as a result of multiple
build commands.
Because multiple build commands may update the same side effect file, by
default the side_effect target is *not* automatically removed when the
target is removed by the -c option. (Note, however, that the side_effect might
be removed as part of cleaning the directory in which it lives.) If you
want to make sure the side_effect is cleaned whenever a specific target is
cleaned, you must specify this explicitly with the Clean
http://www.scons.org/doc/production/HTML/scons-user.html#f-Clean or
env.Clean function.
Reading the description, I think that the SideEffect behavior doesn't cover
the Depends behaviors which Andrew desires.
V/R,
William
On Fri, Oct 31, 2014 at 7:22 PM, Bill Deegan b...@baddogconsulting.com
wrote:
SideEffect() by definition (to me at least) should be informing SCons that
a file is produced, but it is not important. Allowing SCons to be aware of
it.
It seems like using that file as an input in any part of your build would
violate such an idea.
So I think I'm agreeing with Dirk on this one.
Regardless of however the code might be defined today. I think it's
important to be able to tell SCons about such a type of file.
-Bill
On Fri, Oct 31, 2014 at 12:20 PM, Dirk Bächle tshor...@gmx.de wrote:
On 31.10.2014 20:04, Gary Oberbrunner wrote:
On Fri, Oct 31, 2014 at 2:55 PM, Dirk Bächle tshor...@gmx.de wrote:
I don't think there is anything to fix here...and it's no bug for me
either.
Please read the man page for the definition
of a SideEffect, and when it should be used. My understanding is, that
in
your case the conditions do not apply...so you should rather define an
Emitter (see http://www.scons.org/wiki/ToolsForFools for example) that
adds
sf0 and sf1 to the list of targets.
Then the Install() should work as expected.
I disagree. He's explicitly passing the files in sf to Install();
Install() should always (try to) install all the files given as its
sources. Whether they're created as side effects or anything else
_shouldn't_ matter.
Hmm, do you mean all files or all targets? I really don't regard
SideEffects to be full targets...but maybe I'm overlooking something. I'm
just thinking about the case where I use a foo compiler. It always
creates a file named.fix, containing some internal info that's input for
the build step itself.
So, for scheduling single builds with this compiler correctly, in
concurrency speak: locking it with a kind of monitor for parallel mode, I
define the file named.fix as SideEffect in my foo Builder.
If I then say something like:
env = Environment(tools=['foo'])
a = env.Foo('a.foo','a.in')
b = env.Foo('b.foo','b.in')
env.Install('/install', [a, b, 'named.fix'])
which version of named.fix gets installed? Which version *should* get
installed? And is the intended ordering guaranteed for any invocation of
-j n?
You're probably right, but the above is what's rolling through my head
right now...all kinds of clarifications and comments are welcome. ;)
Dirk
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Likely bug - installing side effect files
On Thu, Oct 30, 2014 at 9:34 AM, Ben Golding ben.gold...@synopsys.com wrote:
tgt = Command('tgt', 'src', 'touch $TARGET sf0 sf1')
sf = SideEffect([ 'sf0', 'sf1' ], tgt)
Install('dir', tgt + sf)
I can reproduce this bug. Please file a ticket! The dependency tree
looks OK, and I can't easily see the cause of the bug.
Adding sf0 and sf1 to the targets list instead of as side effects makes it work.
--
Gary
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Likely bug - installing side effect files
Thanks Gary.
http://scons.tigris.org/issues/show_bug.cgi?id=2982
I started to add some additional trace to Taskmaster.py, and compare the logs
from sequential vs. parallel builds.
The bug seems to relate to how the node states are used for the side-effect
nodes:
- In executed_with_callbacks(), the side-effect state is set to NODE_NO_STATE.
- In postprocess(), the logic to decrement the parent ref count is only active
for a side-effect node with state NODE_EXECUTING.
Although this helped me to understand the bug a bit better, I'm not sure what
is the correct fix.
Regards
Ben
-Original Message-
From: Scons-dev [mailto:scons-dev-boun...@scons.org] On Behalf Of Gary
Oberbrunner
Sent: 30 October 2014 14:30
To: SCons developer list
Subject: Re: [Scons-dev] Likely bug - installing side effect files
On Thu, Oct 30, 2014 at 9:34 AM, Ben Golding ben.gold...@synopsys.com wrote:
tgt = Command('tgt', 'src', 'touch $TARGET sf0 sf1')
sf = SideEffect([ 'sf0', 'sf1' ], tgt)
Install('dir', tgt + sf)
I can reproduce this bug. Please file a ticket! The dependency tree
looks OK, and I can't easily see the cause of the bug.
Adding sf0 and sf1 to the targets list instead of as side effects makes it work.
--
Gary
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
