Jason, >From a brief look at this it seems like a Parts specific issue to me. In terms of the solution you are suggesting may make sense in the predefined configuration Parts is supporting. (From what you're saying Parts makes a bunch of decisions on layout of the tree for the users).
Also if this is specifically about your UnitTest builder, change it into a psuedo method and monkey with the source nodes there? -Bill On Fri, Mar 11, 2016 at 6:07 PM, Jason Kenny <[email protected]> wrote: > Bill, > > > > >>SCons only allows one variantdir processed per target dir, but not per > Environment(). > > > > In Parts, each Part has its own variant directory define by the BUILD_DIR > variable. So this is not an issue. Making mapping to # for each part would > make a different directory. > > > > >> How would this work when there are N variants (and thus N variant > directory trees)? > > > > I guess how one uses Variant directories could make this complex. A common > case shown is to use VariantDir to control build configurations, such as > Debug and Release. This could be done in Raw Scons, however I found this > for larger projects very complex to do ( much like using make VPATH is). I > deal with this in Parts via defining a configuration concept that is bound > the toolchain improvements I have made. This allows me to define a CONFIG > variable that can be used per environment (or Part/Component) to control > what setting are loaded. For the VariantDir point of view the CONFIG value > is used in compose BUILD_DIR finial value. This mean the Parts has a > VariantDir for each Part. Adding the idea of out of tree files for the > Sconstruct or a given Parts ( as I have working now ) is just adding a two > new variants for each Parts before I read in the file. The new Variants are > controlled by new BUILD_DIR variable like ROOT_BUILD_DIR which would > control path outside the Sconstruct tree. > > > > For to get this to work I have to do two things: > > 1) Add a AbsDirNode() and AbsFileNode() to correctly translate the > string and returns to the correctly formed Node so Scons will make a > variant node correctly for the source. (I had a AbsDir and AbsFile, but the > return strings and are useful for adding values to variables like CPPATH) > > 2) Tweak my UnitTest function to process certain string correctly > based on the new values > > > > My thought was that it would be nice is this was build into the Node API > to just do this. So if a user said: > > env.File(‘#somedir/sub1/file2’) > > > > it would be mapped to a “$ROOT_BULD_DIR/somedir/sub1/file2” automatically > when making a node. I however understand such a change could have high > risk, and would need to be tested. I also understand that for me in Parts I > sort of have a state because of the BUILD_DIR variables I have setup, so > any Part would have three VariantDir() being “active” at a time because of > what BUILD_DIR, ROOT_BUILD_DIR, etc would map to when a Part file is read. > SCons does not have at the core a component notion of any kind so this > probably would not map well in SCons in hindsight. SCons would need some > “state” API to allow for this in an order dependent way. It might be > possible if the SConscript could be updated to take a list of VariantDir > and src_dir. I would have to test and see what would happen if I tried to > modified the Nodes via a monkey patch in Parts to see if I could have the > Variant “path” translation happen naturally in the nodes in a safe way. > Till then I think you are right in that this does not make sense yet in raw > SCons. > > > > >>You might be better to use Repository pointing targets into the > variantdir? > > I am not sure how this would help. Honestly I was looking at using the > Repository feature to help speed up large build, by allowing the user to > point to a build on a shared drive and build off of that. The idea here is > different from CacheDir() in that it would point to a given build to allow > a developer to quick debug and fix an issue seen in the field without > having to build that revision locally. How did you think Repository() could > help with this VariantDir() discussion? > > > > > > I have a pull request open for my fix in Parts to this issue. I added you > Bill in case you wanted to peak at it and provide any thoughts to what I am > doing in Parts. > > > > Jason > > > > > > > > Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for > Windows 10 > > > > *From: *Bill Deegan <[email protected]> > *Sent: *Thursday, March 10, 2016 12:37 PM > *To: *Jason Kenny <[email protected]> > *Cc: *SCons developer list <[email protected]>; [email protected] > > *Subject: *Re: [Scons-dev] VariantDir() and #paths to files > > > Jason, > > > > > On Thu, Mar 10, 2016 at 12:23 PM, Jason Kenny <[email protected]> wrote: > > Bill, > > > > First, let me add this to user group. > > > > So maybe I am doing something wrong, but.. > > > > I think the core issue here is that SCons only allows one active > VariantDir at a time to be actively translated. > > > SCons only allows one variantdir processed per target dir, but not per > Environment(). > So src/a -> output/a, src/b -> output_b/b, etc is fine > > src/a -> output/a > src/bin -> output/b # Fails.. > > > > When a user does a > > > > Env.VariantDir(“debug”,”src”) > > Env.Sconscript(“debug/Sconscript”) > > > > The user has to refer to files in the Sconscript as “file1” or > “sub1/file2” and these value get changed into “debug/file1” or > “debug/sub1/file2”. > > > > If I was to refer to a file outside the src directory such as a > “src2/file3”. This requires active translation by the user for file not > under the source directory of the sconscript. > > > > I can do one of two things. > > > > 1. add a another VariantDir(“#debug/outoftree”,”.”) and then monkey > path env.File() to auto translate any string that is not under the current > source directory. This way a #src2/file3 would be translated to a > “#debug/outoftree/src2/file3”, much like the file “sub1/file2” is turned in > to “debug/sub1/file2” inside the Sconscript with out user intervention. > > 2. Try to setup funny relative path, however if this tends to fail > if the Variant directory and source directory don’t have the same level of > subdirectories. > > > > I think what would be useful for SCons is to have the logic in the Nodes > to allow for 1) to work. While one can argue about what the current way > Variant Directory works is correct, I believe it is clear that users need > the translation to work for usability. In complex builds it is not always > obvious what the Variant Directory is. > > > How would this work when there are N variants (and thus N variant > directory trees)? > > > > > I think I going to have to setup some form of 1) to address this problem > in Parts. This means I setup a set of 1 or 2 more active directory’s that I > would look for when the File, Dir, etc node is made and translate the path > if it start with correct source path. This will allow different parts such > as PartA and PartB refer to a common file out these parts subtree and built > it correctly for each Part case. > > > > Or I could add a new variable like $ROOTNODE_BUILD_DIR and change the > Parts AbsFile() api to translate these files to correct Variant Directory > based on it being under the Sconstruct directory or outside it. > > > > I think making a way to do this more directly without a monkey patch would > be useful. I need to look at the second option with modifying the Part > AbsFile API to see if this could work. This might be an easy addition that > could be moved to SCons. > > > You might be better to use Repository pointing targets into the variantdir? > > -Bill > > > > Jason > > > > > > Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for > Windows 10 > > > > *From: *Bill Deegan <[email protected]> > *Sent: *Wednesday, March 9, 2016 9:29 PM > *To: *SCons developer list <[email protected]> > *Subject: *Re: [Scons-dev] VariantDir() and #paths to files > > > Jason, > > The problem you are seeing ( believe) is that calling SConscript() with a > variant_dir argument performs the function > > VariantDir(whatever the variant dir was specified, directory of the > SConscript file). > > Where you'd likely need also > > VariantDir(whatever the variant dir was specified, "#/myprog/test/unit") > And then to have the source file referenced via the variant dir and not > the source dir #/myprog/test/unit. > > So then the question would be is it correct to automatically apply the > variantdir behavior to any source file specified in the SConscript being > read when it is specified with variant_dir? > One thing for sure, doing so would be a break with current functionality > and would trigger a deprecation cycle. > > Would it make sense to provide (an)other flag(s) to tell SConscript to do > this? > > These are just some items to consider from a quick look at the issue. > > Also, I'd say this would be safe/better for users mailing list since this > is about how to use/proper functioning of user visible SCons logic. > -Bill > > On Wed, Mar 9, 2016 at 10:15 PM, Jason Kenny <[email protected]> wrote: > > > > Hi guys, > > > > There has been a constant issue I have had with using SCons with > VariantDir logic. I was wondering if I am missing something. > > > > The issue shows up a number of time when people use Parts. The issue is > when the user wants to use a file outside the directory containing the part > file ( From a pure Scons point of view a part file is what would be called > a SConscript). I believe this is a known issue with SCons, however it seems > this is a common problem I have seen many times, enough for me to think > there should be some better solution. > > > > I have a example of this with an issue in Parts just opened. > https://bitbucket.org/sconsparts/parts/issues/1/unit-test-object-build-files-can-appear-in > > > > The easy answer for me is to say move the parts file up to the root. And > tweak a few paths in a part file, to allow all files to be at the same > level or below the location of the parts file. While this will solve the > problem, I know it will be seen again as users have a reason they want to > locate the “build” files in different locations from the all or some of the > source. In this example they seem to have a pattern in which they have some > “common” test helpers files that they want other Parts to use when creating > there unit test. It makes sense, I would like to find a way to allow this > functionality to the user in a painless way ( ie I do something under the > covers 😊 ). There is an attached example of the issue. > > > > Does anyone have any thought on this? > > > > > > Jason > > > > > > > > > > > > Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for > Windows 10 > > > > _______________________________________________ > Scons-dev mailing list > [email protected] > https://pairlist2.pair.net/mailman/listinfo/scons-dev > > > >
_______________________________________________ Scons-dev mailing list [email protected] https://pairlist2.pair.net/mailman/listinfo/scons-dev
