Dirk, An after thought on my original reply:
Users might also accidentally delete "*.class" files in your output dir, > between two builds. And they expect SCons to rebuild those intermediate > targets on its next invocation. How do you want to handle this? SCons wouldn't rebuild since no source had changed. This would be an issue >> if the user was trying to share *.class file(s) which would only be >> relevant with something like SCons.JavaH, but it supports the directory >> parameter(s). SCons could check that the directory contents had changed and >> mark as requiring rebuild; I am grabbing at straws on this one... > > In the case of SCons.JavaH, you would probably want to denormalize the build. This might sound terrible at first, but it work(s) well in some database implementation(s) requiring high-speed throughput. JavaH work make a separate Java compiler invocation. It would require more overall work, but the build would be correct, and the target directory could be cleaned up after Java/JavaH call(s) since the file(s) are waste product(s) of the process and wouldn't be tracked in the DAG. V/R, William On Fri, May 30, 2014 at 7:39 PM, William Blevins <[email protected]> wrote: > Dirk, > > Team, >> I have a couple points I'd like to discuss, but for the sake of >> organization, I intend to split them into separate emails. >> Java Part #2 >> There has been some discussion on making Scons more ANT-like, >> http://www.scons.org/wiki/JavaSupport, and that might solve some issues >> like identifying Java targets. There is merit here assuming that Java >> support stays SCons-like at the same time; otherwise, why not just use ant >> or maven? >> > > > From this I get that you'd like to have some form of Java support in SCons >>> directly, and don't want to delegate things to an external program, >>> correct? At least it's my assumption for the following remarks... >> >> SCons already has Java support (right?). I have been a SCons user for a > few years and am interested in trying to make the Java toolkit better > because I dislike Ant-like build system(s), but I feel that SCons still has > room for improvement. I am braining storming ways to improve the setup. I > have a bunch of idea(s) that I would like to discuss and this just happens > to be one of them (#2 in fact). I know that the SCons teams has talked > about refactoring the SCons Java toolkit, but outside the notes from the > link above, I haven't seen any further discussions. I am the new kid on > the scons-dev mailing list block, so feel free to give me a tour. I would > be willing to work on SCons with some direction. > > Problem(s) this will solve: >> 1. Determining the source -> target(s) from *.java -> *.class is painful, >> possibly java version dependent, and performance intensive (possibly); >> thus, we should optimize this out. >> 2. Java developers are plagued requiring manual cleaning of a jar far too >> often with ant-like system(s) and the current scon(s) methodology. >> 1. Clean could be required when deleting or moving a file since the >> last build output isn't removed otherwise; avoidable in current SCons >> depending on how sources are specified (not obvious and/or intuitive coming >> from ant-like system(s)). >> 3. Simplify the SCons Java API to allow building jar file(s) without >> having to interact with *.class file(s) at all. >> 4. Simplify the SCons Java support for resource file(s); currently, you >> have to do copying by hand (if it's supported I know not where or how). >> 5. Reduce the amount of code required to build a jar (by a lot). >> > > > This sounds like an even more simplified Java Builder than what we have >>> now. If you could come up with a SoSimpleJava() Builder, we'd probably add >>> it to the core and let people have their own choice. >> >> That's fair, but since I am talking about not tracking *.class file(s) at > all. The old function(s) couldn't exist. > > How I imagine this should work: > > 1. Treat class file(s) like side-effect(s) and do not include them (at >> all) in the dependency tree. >> 1. This means we don't have to scan the Java file(s) and determine >> *.class targets. >> > > > I don't think that you can have both. If you treat files as SideEffects >>> (in the SCons sense), they have to be part of the dependency DAG. See next >>> comment... >> >> I meant this in the truer form of side-effect and not the > SCons.SideEffects sense. Getting from A -> C by passing through B, but B > is just along the way. The only way the *.class file(s) could be part of > the DAG is if they were identified after the Java compiler had already > created the *.class file(s) by basically Glob(*.class). > > I would really like to get away from SCons being required to scan Java > source file(s) to determine the targets due to the 1 -> N nature of *.java > to *.class. This would resolve several issues with the SCons Java toolkit > including incorrect dependency tree(s) with derived (generated) Java source > file(s) and some build issue(s) that require manual clean operation(s) on > Java code (which may be alleviated depending on how you setup your > SConscript, but may not be obvious coming from an ant-like system). > > The Java DAG is already incomplete because SCons doesn't add item(s) in > JAVACLASSPATH automatically, so user(s) are required to do it manually > anyway. At first I thought this was a bug, but it's even documented in the > User's Guide, so I assume it's intentional which confuses the crap out of > me? > > 2. Remove the Java(...) function from SCons; thus, do not interact with >> *.class file(s) directly ever. There isn't a *good* reason to do this. >> > > > Here is just one scenario, where dealing with *.class files actually makes >>> sense: If you're using anonymous classes in a Java source, a bunch of >>> additional class files get generated. How do you remove these files on a >>> "clean", if they are located in a source folder (which people might prefer >>> for some reason)? >> >> Remove all *.class file(s) generated within the directory tree were the > Java compiler is expected to output it's source. I'd almost prefer to > require the build directory and just delete it. Making the build > configuration simple and clean is more important I think. > > More experienced Java developers can probably come up with a dozen more >>> useful cases...I'm not sure how often people would want to integrate >>> pre-built *.class files (coming from Scala perhaps?) into their builds, for >>> example. >> >> Are you talking about coping *.class file(s) between build directories? > Why not use a *.jar file? At that point you are just moving code > copy-pasta from your source code to your build tool. > > You may be able to work around the "clean" problem, by creating all >>> "*.class" files in a separate folder structure (pretty much like a >>> VariantDir). But what you'll end up with is a somewhat crippled DAG, >>> forcing you to remove this output dir completely and rebuild everything >>> from scratch...even when only a comment in a single Java source is changed. >>> Only to "be sure", just like in make or Ant...there is not much of a win >>> here for the user, I'm afraid. >> >> The win here is that Java compiler is much faster than the SCons source > emitter (I think that's the right term) for Java. There is a performance > bottleneck(s) in the Java toolkit which I will discuss in another email; I > hope the team is alright with email attachments because I have performance > logs. > > Users might also accidentally delete "*.class" files in your output dir, >>> between two builds. And they expect SCons to rebuild those intermediate >>> targets on its next invocation. How do you want to handle this? >> >> SCons wouldn't rebuild since no source had changed. This would be an > issue if the user was trying to share *.class file(s) which would only be > relevant with something like SCons.JavaH, but it supports the directory > parameter(s). SCons could check that the directory contents had changed and > mark as requiring rebuild; I am grabbing at straws on this one... > > Finally, you should take into account that users might want to create >>> different versions of the same source code. Just as an example, let's >>> assume they want to create an "original" and "obfuscated" target(s), or a >>> "simple" and "extended" release where additional packages get included >>> based on environment variables. And for maximum speedup they do this in >>> parallel, by using the "-j" option...so a single dedicated output dir won't >>> do probably. >> >> There would be 1 target directory per Java compiler invocation which I > imagined would be 1 target directory per *.jar created. The gain from > threading packages in a *.jar file shouldn't be observable until well into > 1000+ *.class file(s) per *.jar file. At this point, I imagine there are > bigger architecture concerns at hand than the compile time of 1 *.jar file. > > V/R, > William > > > > On Fri, May 30, 2014 at 7:13 AM, Dirk Bächle <[email protected]> wrote: > >> Hi William, >> >> please find a few comments below. >> >> >> On 30.05.2014 02:42, William Blevins wrote: >> >> Team, >> >> I have a couple points I'd like to discuss, but for the sake of >> organization, I intend to split them into separate emails. >> >> Java Part #2 >> >> There has been some discussion on making Scons more ANT-like, >> http://www.scons.org/wiki/JavaSupport, and that might solve some issues >> like identifying Java targets. There is merit here assuming that Java >> support stays SCons-like at the same time; otherwise, why not just use ant >> or maven? >> >> From this I get that you'd like to have some form of Java support in >> SCons directly, and don't want to delegate things to an external program, >> correct? At least it's my assumption for the following remarks... >> >> >> Problem(s) this will solve: >> 1. Determining the source -> target(s) from *.java -> *.class is painful, >> possibly java version dependent, and performance intensive (possibly); >> thus, we should optimize this out. >> 2. Java developers are plagued requiring manual cleaning of a jar far too >> often with ant-like system(s) and the current scon(s) methodology. >> 1. Clean could be required when deleting or moving a file since the >> last build output isn't removed otherwise; avoidable in current SCons >> depending on how sources are specified (not obvious and/or intuitive coming >> from ant-like system(s)). >> 3. Simplify the SCons Java API to allow building jar file(s) without >> having to interact with *.class file(s) at all. >> 4. Simplify the SCons Java support for resource file(s); currently, you >> have to do copying by hand (if it's supported I know not where or how). >> 5. Reduce the amount of code required to build a jar (by a lot). >> >> This sounds like an even more simplified Java Builder than what we >> have now. If you could come up with a SoSimpleJava() Builder, we'd probably >> add it to the core and let people have their own choice. >> >> How I imagine this should work: >> 1. Treat class file(s) like side-effect(s) and do not include them (at >> all) in the dependency tree. >> 1. This means we don't have to scan the Java file(s) and determine >> *.class targets. >> >> I don't think that you can have both. If you treat files as SideEffects >> (in the SCons sense), they have to be part of the dependency DAG. See next >> comment... >> >> >> 2. Remove the Java(...) function from SCons; thus, do not interact with >> *.class file(s) directly ever. There isn't a *good* reason to do this. >> >> Here is just one scenario, where dealing with *.class files actually >> makes sense: If you're using anonymous classes in a Java source, a bunch of >> additional class files get generated. How do you remove these files on a >> "clean", if they are located in a source folder (which people might prefer >> for some reason)? >> >> More experienced Java developers can probably come up with a dozen more >> useful cases...I'm not sure how often people would want to integrate >> pre-built *.class files (coming from Scala perhaps?) into their builds, for >> example. >> >> You may be able to work around the "clean" problem, by creating all >> "*.class" files in a separate folder structure (pretty much like a >> VariantDir). But what you'll end up with is a somewhat crippled DAG, >> forcing you to remove this output dir completely and rebuild everything >> from scratch...even when only a comment in a single Java source is changed. >> Only to "be sure", just like in make or Ant...there is not much of a win >> here for the user, I'm afraid. >> >> Users might also accidentally delete "*.class" files in your output dir, >> between two builds. And they expect SCons to rebuild those intermediate >> targets on its next invocation. How do you want to handle this? >> >> Finally, you should take into account that users might want to create >> different versions of the same source code. Just as an example, let's >> assume they want to create an "original" and "obfuscated" target(s), or a >> "simple" and "extended" release where additional packages get included >> based on environment variables. And for maximum speedup they do this in >> parallel, by using the "-j" option...so a single dedicated output dir won't >> do probably. >> >> Best regards, >> >> Dirk >> >> >> _______________________________________________ >> Scons-dev mailing list >> [email protected] >> http://two.pairlist.net/mailman/listinfo/scons-dev >> >> >
_______________________________________________ Scons-dev mailing list [email protected] http://two.pairlist.net/mailman/listinfo/scons-dev
