I also checked scons 2.4.1 and 2.3.0, so this goes back a while apparently? Or maybe this never worked right...
Can someone confirm? V/R, William On Tue, Mar 21, 2017 at 6:09 PM, William Blevins <[email protected]> wrote: > Moving to scons-dev list. Notice email change ^ > > When I look at the "sconsign -c .scons.dblite ", I noticed that the source > file does not have a csig, so the default decider will not cause a rebuild > (MD5-only). > > $ sconsign .sconsign.dblite >> === .: >> SConstruct: None 1490133556 120 >> permissions: None 1490133631 5 >> === /bin: >> chmod: None 1487766225 60296 >> mkdir: None 1487766225 81032 >> > > The dependency tree looks right: > > $ scons --tree=all,status >> scons: Reading SConscript files ... >> scons: done reading SConscript files. >> scons: Building targets ... >> scons: `.' is up to date. >> E = exists >> R = exists in repository only >> b = implicit builder >> B = explicit builder >> S = side effect >> P = precious >> A = always build >> C = current >> N = no clean >> H = no cache >> >> [E b C ]+-. >> [E C ] +-SConstruct >> [E B C ] +-aaa >> [E C ] | +-permissions >> [E C ] | +-/bin/mkdir >> [E C ] | +-/bin/chmod >> [E C ] +-permissions >> > > Even if I change the decider I still don't get a rebuild: > > env = DefaultEnvironment() >> >> env.Decider('MD5-timestamp') >> env.Command(Dir('aaa'), File('permissions'), ['mkdir $TARGET', 'chmod >> `cat permissions` $TARGET']) >> > > Information: > > $ scons --version >> SCons by Steven Knight et al.: >> script: v2.5.1.rel_2.5.1:3735:9dc6cee5c168[MODIFIED], 2016/11/03 >> 14:02:02, by bdbaddog on mongodog >> engine: v2.5.1.rel_2.5.1:3735:9dc6cee5c168[MODIFIED], 2016/11/03 >> 14:02:02, by bdbaddog on mongodog >> engine path: ['/usr/lib/scons/SCons'] >> Copyright (c) 2001 - 2016 The SCons Foundation >> > >> > $ python --version >> > Python 2.7.13 >> > > SCons distribution per Debian Apt 2.5.1-1 > > Thoughts? > William > > > > > > > On Tue, Mar 21, 2017 at 5:39 PM, William Blevins <[email protected]> > wrote: > >> I see what you are saying. I need to look into it some. Did this work in >> a previous version of SCons? What version did it stop working in? What >> version of Python are you using? >> >> It's a bit strange. I needed to fix your example, but I got it to >> replicate. At a glance, the dependency tree looks correct, but nothing >> happens when the permissions file is updated. >> >> V/R, >> William >> >> On Tue, Mar 21, 2017 at 2:09 PM, Manish Vachharajani < >> [email protected]> wrote: >> >>> Let me try to get the discussion back to my main point. There is a bug >>> in scons and there should be a ticket for it and a fix (unless I've >>> misunderstood something, which from the discussion I don't think I have). >>> The nature of the bug is this: >>> >>> 1) The user tells scons that there is a target (foo in my example) >>> 2) The target depends on a source (foo-contents.txt in my example) >>> 3) When the target is out of date with respect to the source, rerun the >>> builder >>> >>> It isn't scons' job, in my opinion, to second guess me and decide that >>> it doesn't really need to run the builder, or that the target isn't the >>> precise output set of the builder etc. The point is that there is a target >>> which is out of date with respect to the source and scons is not running >>> the builder. Everything else, IMHO, is an aside. Does that make it >>> clear? I'm not trying to be rude but something is getting lost when I'm >>> trying to be indirect. >>> >>> I'm glad to discuss what I believe is the root cause based on my >>> investigations in the python debugger. 99% of the time this doesn't matter >>> because there is almost never a directory only set of targets with a >>> builder that isn't simply a mkdir builder or equivalent. Note, however, if >>> the builder were something like: >>> >>> env.Command(['/a/b/c'], >>> ['permissions.txt'], >>> [mkdir $TARGET, >>> chmod `permissions.txt` $TARGET]) >>> >>> scons would fail to update the permissions on the directory when >>> permissions.txt changed if /a/b/c already existed. I hope that makes it >>> more clear the exact nature of the issue. My example is simple because it >>> is the smallest SConstruct I could create to demonstrate the issue. >>> >>> For my specific case, I have workarounds, and a custom builder that >>> uses them. A custom builder doesn't help without the workarounds because >>> the problem is in the up to date logic, not the scanner and action logic in >>> scons. Oh, and while package managers let you see content, it doesn't help >>> if the package is going to fetched from the network and I need to calculate >>> dependencies before the package is fetched. >>> >>> Manish >>> >>> >>> >>> On Tue, Mar 21, 2017 at 10:53 AM, William Blevins <[email protected] >>> > wrote: >>> >>>> Manish, >>>> >>>> Ah, ok. I was confused when your original example seemed so simple. You >>>> should probably create a custom builder then: >>>> https://bitbucket.org/scons/scons/wiki/ToolsForFools >>>> >>>> It isn't a bug that SCons doesn't rebuild targets it doesn't have. The >>>> issue is that you need to tell SCons about said targets. Do this via a >>>> custom builder. >>>> >>>> I haven't used npm, but most package managers have a way to list the >>>> contents of a package, so just convert that list into a target set. I would >>>> need more information about the source -> target dependency chain in terms >>>> of actual vs expected in order to say much more. >>>> >>>> Sorry for the fix response and run, but normally I don't get time to >>>> answer emails this early in the day. Let us know if you think a custom >>>> builder will not solve your problem. >>>> >>>> V/R, >>>> William >>>> >>>> On Tue, Mar 21, 2017 at 12:22 PM, Manish Vachharajani < >>>> [email protected]> wrote: >>>> >>>>> If you look at my original message, the problem is that in my real use >>>>> case, I have no idea what the file targets will be. The only target I >>>>> know >>>>> will be built by the builder is a set of directories. Your example simply >>>>> causes scons to have a file target in the second Command builder which >>>>> works around the bug as I describe in my original email. In real life, my >>>>> builder looks something like this; >>>>> >>>>> env.Command(['node_modules/minimist', 'node_modules/@types/node, >>>>> ...], #these are directories >>>>> ['package.json'], >>>>> ['npm install --global-style'], >>>>> chdir=Dir('.')) #chdir because npm doesn't have a way to >>>>> install elsewhere >>>>> >>>>> In this case, I have no idea what files npm install will create. The >>>>> targets array is actually created by reading package.json and looking at >>>>> the dependencies and devDependencies fields of the main object, so those >>>>> aren't even hard-coded in the SConstruct/SConscript. >>>>> >>>>> Again, if I fake out scons by have a 'node_modules/.built' File target >>>>> in the list and add a 'touch node_modules/.built' command to the builder >>>>> after 'npm install', all will work as expected because there is a File >>>>> target. In your example, things work because the second builder has a >>>>> file >>>>> target and SCons will only run the first builder in your example when foo >>>>> does not exist, regardless of the state of foo-contents.txt in relation to >>>>> what is in .sconsign. That is the essence of the bug, if the target >>>>> directory exists and it is the only target, the builder will not run >>>>> regardless of the state of the sources. >>>>> >>>>> Manish >>>>> >>>>> >>>>> On Mon, Mar 20, 2017 at 9:09 PM, William Blevins < >>>>> [email protected]> wrote: >>>>> >>>>>> Manish, >>>>>> >>>>>> * Why does my example not work? >>>>>> >>>>>> SCons Comand Builder parameters are target(s), source(s), command(s): >>>>>> http://scons.org/doc/HTML/scons-user.html#chap-builders-commands >>>>>> >>>>>> Foo is the target parameter, and foo-contents.txt is the source >>>>>> parameter. Targets explicitly depend on sources. In your example, >>>>>> directory >>>>>> foo depends on source file foo-contents.txt; target file copy does not >>>>>> depend on source file. >>>>>> >>>>>> * How do I fix this issue? >>>>>> >>>>>> 1. Call Command Builder correctly: Command('foo', 'foo-contents', >>>>>> 'mkdir $TARGET') which creates file dependency on directory, and then >>>>>> Command('foo/foo-contents.txt', 'foo-contents.txt', 'cp $SOURCE >>>>>> $TARGET') which creates file target dependency on file source. >>>>>> >>>>>> 2: [PREFERRED] Use the SCons FileSystem factory >>>>>> http://scons.org/doc/HTML/scons-user.html#chap-factories >>>>>> >>>>>> V/R, >>>>>> William >>>>>> >>>>>> >>>>>> On Mon, Mar 20, 2017 at 1:47 PM, Manish Vachharajani < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> I believe there is a bug in how up to date is computed for file >>>>>>> nodes and directory targets. Below is a SConstruct and run output >>>>>>> illustrating the bug. >>>>>>> >>>>>>> ############## >>>>>>> # SConstruct >>>>>>> env = Environment() >>>>>>> targets = env.Command('foo', >>>>>>> 'foo-contents.txt', >>>>>>> ['mkdir foo', >>>>>>> 'cp foo-contents.txt foo']) >>>>>>> env.Default(targets) >>>>>>> >>>>>>> ############## >>>>>>> # Shell log >>>>>>> >>>>>>> # Create the file that will be installed into foo/foo-contents.txt >>>>>>> $ echo "Hello, " >> foo-contents.txt >>>>>>> # All is well, scons runs the command builder as expected >>>>>>> $ scons >>>>>>> >>>>>>> scons: Reading SConscript files ... >>>>>>> scons: done reading SConscript files. >>>>>>> scons: Building targets ... >>>>>>> mkdir foo >>>>>>> cp foo-contents.txt foo >>>>>>> scons: done building targets. >>>>>>> >>>>>>> # All is well, the target is up to date as expected >>>>>>> $ scons >>>>>>> >>>>>>> scons: Reading SConscript files ... >>>>>>> scons: done reading SConscript files. >>>>>>> scons: Building targets ... >>>>>>> scons: `foo' is up to date. >>>>>>> scons: done building targets. >>>>>>> >>>>>>> # Uh-oh, source is updated but the target is considered up to date! >>>>>>> $ echo "World!" >> foo-contents.txt >>>>>>> $ scons >>>>>>> >>>>>>> scons: Reading SConscript files ... >>>>>>> scons: done reading SConscript files. >>>>>>> scons: Building targets ... >>>>>>> scons: `foo' is up to date. >>>>>>> scons: done building targets. >>>>>>> >>>>>>> For the bug to trigger, all targets for a builder must be >>>>>>> directories. The target directory must exist and the source file must >>>>>>> exist and be out of date due to a content update (resulting in a >>>>>>> signature >>>>>>> mismatch). >>>>>>> >>>>>>> I think the problem is that is_up_to_date for File nodes only checks >>>>>>> the signatures of its children, which avoids this bug as long as one of >>>>>>> the >>>>>>> builder targets is a File and not a Directory. However, I haven't done >>>>>>> enough analysis to be sure that this is the issue. >>>>>>> >>>>>>> A work around is to make sure that you have a file (or create a >>>>>>> dummy .built file) and include it as a target. >>>>>>> >>>>>>> This is showing up specifically for us when trying to calculate >>>>>>> dependencies for a package manager (npm) where we have no insight into >>>>>>> the >>>>>>> contents of the resulting directory. >>>>>>> >>>>>>> Manish >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Scons-users mailing list >>>>>>> [email protected] >>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users >>>>>>> >>>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Scons-users mailing list >>>>>> [email protected] >>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users >>>>>> >>>>>> >>>>> >>>>> _______________________________________________ >>>>> Scons-users mailing list >>>>> [email protected] >>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users >>>>> >>>>> >>>> >>>> _______________________________________________ >>>> Scons-users mailing list >>>> [email protected] >>>> https://pairlist4.pair.net/mailman/listinfo/scons-users >>>> >>>> >>> >>> _______________________________________________ >>> Scons-users mailing list >>> [email protected] >>> https://pairlist4.pair.net/mailman/listinfo/scons-users >>> >>> >> >
_______________________________________________ Scons-dev mailing list [email protected] https://pairlist2.pair.net/mailman/listinfo/scons-dev
