William,
thanks for your quick reply.
On 31.05.2014 01:39, William Blevins wrote:
Dirk,
Team,
I have a couple points I'd like to discuss, but for the sake of
organization, I intend to split them into separate emails.
Java Part #2
There has been some discussion on making Scons more ANT-like,
http://www.scons.org/wiki/JavaSupport, and that might solve some
issues like identifying Java targets. There is merit here
assuming that Java support stays SCons-like at the same time;
otherwise, why not just use ant or maven?
From this I get that you'd like to have some form of Java
support in SCons directly, and don't want to delegate things
to an external program, correct? At least it's my assumption
for the following remarks...
SCons already has Java support (right?). I have been a SCons user for
a few years and am interested in trying to make the Java toolkit
better because I dislike Ant-like build system(s), but I feel that
SCons still has room for improvement. I am braining storming ways to
improve the setup. I have a bunch of idea(s) that I would like to
discuss and this just happens to be one of them (#2 in fact). I know
that the SCons teams has talked about refactoring the SCons Java
toolkit, but outside the notes from the link above, I haven't seen any
further discussions. I am the new kid on the scons-dev mailing list
block, so feel free to give me a tour. I would be willing to work on
SCons with some direction.
That's good to hear, any helping hand is welcome. I haven't been
involved in the latest Java development, so the link list I gave you in
the mailing thread
http://four.pairlist.net/pipermail/scons-users/2014-February/002225.html
is all I got.
Problem(s) this will solve:
1. Determining the source -> target(s) from *.java -> *.class is
painful, possibly java version dependent, and performance
intensive (possibly); thus, we should optimize this out.
2. Java developers are plagued requiring manual cleaning of a jar
far too often with ant-like system(s) and the current scon(s)
methodology.
1. Clean could be required when deleting or moving a file
since the last build output isn't removed otherwise; avoidable in
current SCons depending on how sources are specified (not obvious
and/or intuitive coming from ant-like system(s)).
3. Simplify the SCons Java API to allow building jar file(s)
without having to interact with *.class file(s) at all.
4. Simplify the SCons Java support for resource file(s);
currently, you have to do copying by hand (if it's supported I
know not where or how).
5. Reduce the amount of code required to build a jar (by a lot).
This sounds like an even more simplified Java Builder than
what we have now. If you could come up with a SoSimpleJava()
Builder, we'd probably add it to the core and let people have
their own choice.
That's fair, but since I am talking about not tracking *.class file(s)
at all. The old function(s) couldn't exist.
How I imagine this should work:
1. Treat class file(s) like side-effect(s) and do not include
them (at all) in the dependency tree.
1. This means we don't have to scan the Java file(s) and
determine *.class targets.
I don't think that you can have both. If you treat files as
SideEffects (in the SCons sense), they have to be part of the
dependency DAG. See next comment...
I meant this in the truer form of side-effect and not the
SCons.SideEffects sense. Getting from A -> C by passing through B,
but B is just along the way. The only way the *.class file(s) could
be part of the DAG is if they were identified after the Java compiler
had already created the *.class file(s) by basically Glob(*.class).
I would really like to get away from SCons being required to scan Java
source file(s) to determine the targets due to the 1 -> N nature of
*.java to *.class. This would resolve several issues with the SCons
Java toolkit including incorrect dependency tree(s) with derived
(generated) Java source file(s) and some build issue(s) that require
manual clean operation(s) on Java code (which may be alleviated
depending on how you setup your SConscript, but may not be obvious
coming from an ant-like system).
Testing out this idea looks like a straightforward task to me. I'd
really like to encourage you to just go ahead and implement a new
Tool/Builder in parallel to the existing ones. Then it's easier to
discuss dis-/advantages and point out the problems of each approach.
However, there certainly will be more obstacles ahead that I can't think
of right now. It's a perfectly understandable reaction to restrict
functionality, such that certain issues don't even emerge in first
place. But one has to be careful about not driving this too far, because
then it's actually more "problem avoidance" than "problem solving".
In fact, you aren't really dealing with issue number 2.) from your own
list of "problems this will solve" above. One could argue that you're
even making it worse, by automating the "make clean; make all" step. So,
it's *guaranteed* that SCons will rebuild the whole JAR on each single
source file change...and the user can't do anything to prevent this.
SCons' way of handling builds (in C/C++ for example) is usually to
prevent the rebuild of intermediate results as much as possible. So be
aware that you're somewhat swimming against the stream with your idea... ;)
[...]
You may be able to work around the "clean" problem, by
creating all "*.class" files in a separate folder structure
(pretty much like a VariantDir). But what you'll end up with
is a somewhat crippled DAG, forcing you to remove this output
dir completely and rebuild everything from scratch...even
when only a comment in a single Java source is changed. Only
to "be sure", just like in make or Ant...there is not much of
a win here for the user, I'm afraid.
The win here is that Java compiler is much faster than the SCons
source emitter (I think that's the right term) for Java. There is a
performance bottleneck(s) in the Java toolkit which I will discuss in
another email; I hope the team is alright with email attachments
because I have performance logs.
I'd really like to have a look at your performance measurements. Perhaps
you can push them up to a public repo, like I did for
https://bitbucket.org/dirkbaechle/scons_testresults ? Then you can
publish your full set of data (plus supporting scripts and whatever else
would be required to reproduce your results), and just have to pass the
URL around.
Users might also accidentally delete "*.class" files in your
output dir, between two builds. And they expect SCons to
rebuild those intermediate targets on its next invocation.
How do you want to handle this?
SCons wouldn't rebuild since no source had changed. This would be an
issue if the user was trying to share *.class file(s) which would only
be relevant with something like SCons.JavaH, but it supports the
directory parameter(s). SCons could check that the directory contents
had changed and mark as requiring rebuild; I am grabbing at straws on
this one...
Finally, you should take into account that users might want
to create different versions of the same source code. Just as
an example, let's assume they want to create an "original"
and "obfuscated" target(s), or a "simple" and "extended"
release where additional packages get included based on
environment variables. And for maximum speedup they do this
in parallel, by using the "-j" option...so a single dedicated
output dir won't do probably.
There would be 1 target directory per Java compiler invocation which I
imagined would be 1 target directory per *.jar created. The gain from
threading packages in a *.jar file shouldn't be observable until well
into 1000+ *.class file(s) per *.jar file. At this point, I imagine
there are bigger architecture concerns at hand than the compile time
of 1 *.jar file.
It could also be several JARs, that the user tries to build in parallel.
So you'll have to ensure that the single output dirs don't ever clash,
by using temporary names for example.
Regards,
Dirk
_______________________________________________
Scons-dev mailing list
[email protected]
http://two.pairlist.net/mailman/listinfo/scons-dev
