Sunday, January 1, 2017, 8:16:42 AM, Taher Alkhateeb wrote:
> Hi Daniel and all,
>
> Okay, I took a look at the ant build script in branch 2.3-gae of the main
> git repository in
> https://git-wip-us.apache.org/repos/asf/incubator-freemarker.git. I have
> the following comments to make
>
> - First, over all I think it is very doable to make a migration. I did not
> find anything to be exceptionally difficult.
> - I think your project might benefit from a multi-project architecture to
> wire the projects together. This would make your life easier as you declare
> dependencies on artifacts rather than projects. More info is here
> https://docs.gradle.org/current/userguide/multi_project_builds.html
I assume you end up with one jar per subproject there, just like with
multi-module Maven projects. That's something we plan to do for
FreeMarker 3 (regardless of the build tool), but for FreeMarker 2 the
final output must remain a single monolithic jar for backward
compatibility. Or, maybe you can do a such multi-jar build and then
repackage the jar-s into one? I don't yet see what ramifications of
that would be (thinking of the Maven source artifacts mess and such).
But for one, it could make be transition to FM 3 smoother too.
> - Some of your tasks are too customized. There are multiple solutions to
> quickly migrate to Gradle. One solution is to keep these tasks in build.xml
> and import them from the gradle script. Another is to rewrite these tasks
> in Gradle but using the Ant DSL (embedded Ant on Gradle). Both of these
> solutions to me are somewhat more preferable than rewriting completely in
> pure gradle.
There's no time pressure in doing this migration, so the cleaner
solution is preferred over the faster one. Considering that only the
output matters, I'm hoping that most customizations can be recreated
in "pure" Gradle. (I count merely calling a custom Ant task like
docgen as "pure".)
> - Another idea that comes to mind is to perhaps break the main repo to
> multiple projects? (not repositories, but projects). The reason is that I
> see dependencies on generated artifacts (e.g. JavaCC artifacts before
> build?).
How that differs from multi-project architecture mentioned earlier? As
of JavaCC, I thought that's just Java code generator plugin that
automatically takes part in the building.
> - Some tasks seem to be unneeded. For example, update-deps is not needed in
> Gradle because you can write the version dependencies in a special format
> that fetches the latest version by default. So every single time you run
> the build system it will check and make sure the latest version is
> downloaded to the gradle cache.
Ant+Ivy does the same, at least as we use it (i.e, with
ivy:cachepath). That we still have an update-deps is a hack to create
an Ivy cache that happens to be stored inside the project directory.
This was a hack to work around some reliability issues with Ivy
offline mode (which perhaps works fine nowadays... it was added many
years ago). Obviously, you can ignore that task. I also hope that we
can avoid Ivy altogether and just rely on the local Maven installation
of the developer (or on its local Maven cache at least... not sure how
it works with Gradle).
> Another unneeded task is init (all automatic stuff in Gradle)
Of course any internal tasks can go without question.
> - Gradle does not download dependencies into the project, but rather keeps
> them in the gradle cache (a special directory to hold all jars e.g.
> $HOME/.gradle). We can create a command to copy these dependencies into the
> project itself. But this might be an area where you can start thinking
> about change. Using gradle cache means the libs disappear from your tree
> but you can refer to them in the IDE and in the classpath automatically
> through gradle.
Agreed.
Other than the offline mode hack mentioned earlier, we also do
explicit dependency downloading to collect the IDE dependencies. That
was done because the Eclipse Ivy plugin had some random problems
sometimes, so we fall back to the primitive but rock solid solution.
So it was yet another hack to avoid problems elsewhere. Hopefully we
won't need it with Eclipse+Gradle...
Talking of IDE support, note that we have some further ugliness there,
as we depend on multiple versions of the same libraries. We depend on
multiple Servlet/JSP versions, Jython versions, but also on multiple
Java versions (though since we have raised minimum required version to
Java 5, you can almost get rid of the last complication, but this can
change again if we want to support Java 8+ stuff later). So we have
just chosen a single dependency version for the IDE dependencies, then
in the Eclipse setup (see in the README) you have to exclude the java
files that depend on different versions. A multi-project setup might
be able to overcome that problem, but then again, the final output
must be a single jar, at least in FreeMarker 2.
> I would like to help out in terms of code contribution, but I'm not very
> familiar with the project. Can you folks help me out with some info,
> specifically:
>
> - Are there any deprecated / unused tasks in build.xml or is this thing
> fully utilized.
None, but as part of this transition we should get rid of the
Ivy-related trickery ("update-deps", "publish-override",
"unpublish-override", "uninstall", etc.). Also we should drop the
functionality "server-publish-last-build"; release engineers should
just install stuff into their local Maven repo (if that's that Gradle
will use), rather then "release" them to a common repository. It's
simpler that way.
> - I see also a lot of properties, are those mostly used by the tasks
> themselves, or do you have things needed by CI? This info is important for
> the design of the new tasks.
By the tasks themselves. (We don't have CI for many years now. We
should have CI though, but that's for another day.)
> - Is the main purpose of the build system to generate artifacts? Do you
> need to replicate the artifacts exactly the same way, or just some end
> artifacts and the middle steps are just a design choice?
Only the final results matter, which are:
- freemarker.jar - monolithic (in 2.x.x at least), and note the things
under the META-INF, and the substitutions in version.properties.
- Maven source and javadoc artifacts
- Manual (HTML genereated via docgen from the XDocBook file)
- JavaDoc (with some class exclusions and with Java 8's typographic
nonsense fixed)
- Apache release artifacts (src and bin, signed and all, as usual).
Note that the bin release contains the generated Manual and JavaDoc.
- Uploading release into the Maven staging repository
- Rat report
- Ensure that the project works with Eclipse fine (and with IntelliJ
ideally, but frankly I don't even know if the current one works with
that well). In the README there's a detailed description of how to
set up Eclipse, where you can describe the necessary steps.
> - Finally, how can we start? What are the next steps? Should I just submit
> a patch of the first round of work for example?
I think the best is if you create a Git fork and in that a feature
branch (see earlier mail), and commit as often as you like. Then we
can discuss things as they evolve, and can merge (or just manually
copy over things) at the end.
> If yes, what is the most important task (target in Ant lingo) to
> start with?
As we can keep using the Ant build until the new one is fully
polished, do it as you like. I guess producing an identical
freemarker.jar is a good start, but it's up to you.
> Cheers,
>
> Taher Alkhateeb
>
>
> On Sun, Jan 1, 2017 at 12:23 AM, Daniel Dekany <ddek...@freemail.hu> wrote:
>
>> Hello,
>>
>> It would be great if you can look at the Ant build and tell your
>> opinion about transforming the same functionality to Gradle. By same
>> functionality I mean getting nearly identical "artifacts" at the end,
>> otherwise everything can change. (For example, you will find that
>> there are some tricky things going on with Ivy; we might not need that
>> anymore.) And of course, you are welcome to actually do the
>> transition.
>>
>> If you look at http://freemarker.org/sourcecode.html, you will find
>> that we have 3 projects. The topic is now the "freemarker" project
>> (and there the work should be done in a feature branch coming out from
>> the "2.3-gae" branch). "freemarker-docgen" and "freemarker-site",
>> which also run on Ant, could be migrated separately, later.
>>
>>
>> Saturday, December 31, 2016, 7:38:32 PM, Taher Alkhateeb wrote:
>>
>> > Hello Everyone,
>> >
>> > We recently switched Apache OFBiz from Ant to Gradle and it was an
>> > amazing experience. I was the main developer working on the transition
>> > and I think if something so complex and wired up with hundreds of
>> > libraries like OFBiz can do it, then I think it would be a much more
>> > pleasant experience with FreeMarker.
>> >
>> > Transitioning to Gradle, however, requires a shift of the way you think,
>> > it is not an imperative style of "do this then do that" and it has
>> > multiple build phases (initialization, configuration and execution).
>> > Gradle is also subject to frequent changes more than ant or Maven. For
>> > example, in versions 3+ the left shift operator "<<" was deprecated even
>> > though heavily used in projects and is subject for total removal in
>> > version 5.
>> >
>> > Overall, I would say between the three build systems, Gradle is the most
>> > powerful hands down as it mixes the convention-over-configuration
>> > philosophy of maven while at the same time allowing overriding any
>> > configuration setting. It also provides a powerful DSL on top of a full
>> > programming language (Groovy)
>> >
>> > If you folks would like to proceed with the transition, and need extra
>> > hands, I'd love to help however I can.
>> >
>> > Cheers,
>> >
>> > Taher Alkhateeb
>> >
>> >
>> > On 2016-12-31 02:43 (+0300), Daniel Dekany <d...@freemail.hu> wrote:
>> >> Friday, December 30, 2016, 11:44:35 PM, Apache wrote:>
>> >>
>> >> > I\u2019m not sure what \u201cfuture-proof\u201d means.>
>> >>
>> >> By that I mean that if we have to do something non-standard, it's>
>> >> probably easier to achieve with Gradle.>
>> >>
>> >> Also, if we include FM2 in this, then we surely has to do non-standard>
>> >> things. In FM2 the resulting freemarker.jar has to be the same>
>> >> monolithic jar, etc.>
>> >>
>> >> Last not least, if someone is interested in Gradle, and so wants to>
>> >> tackle this task, then I believe we should just go for Gradle.>
>> >> Especially as it can be started right now with FM2, while with Maven,>
>> >> he had to wait until FM3 gets somewhere.>
>> >>
>> >> -- >
>> >> Thanks,>
>> >> Daniel Dekany>
>> >>
>> >>
>> >> > Maven is basically the standard the other build tools have to be>
>> >> > compatible with. The Maven team has been discussing ways to enhance>
>> >> > the pom.xml to open up new features for quite some time and do seem>
>> >> > to be making some progress.>
>> >> >>
>> >> > The big downside with an Ant build is that everything is custom.>
>> >> > Maven takes the opposite approach - everything is standard, while>
>> >> > Gradle is somewhere in the middle.>
>> >> >>
>> >> > Ralph>
>> >> >>
>> >> >> On Dec 30, 2016, at 2:05 PM, Daniel Dekany <dd...@freemail.hu>
>> wrote:>>
>> >> >> >
>> >> >> I think recreating the Ant build of FM 2 in Gradle (with nearly>
>> >> >> identical output) would be a good start. For starters, it's useful
>> in>>
>> >> >> itself, like it makes FM 2 look less ancient. (-; It's parhaps also>
>> >> >> useful if we want to become Android friendly. Then, it will be
>> easier>>
>> >> >> to proceed with a modular Gradle build in FM 3. (I'm leaning
>> towards>
>> >> >> Gradle as opposed to Maven because it gives us more flexibility, so>
>> >> >> it's more future proof. As history has shown, FreeMarker often
>> needs>
>> >> >> that.)>
>> >> >> >
>> >> >> If you or anyone else feels like playing with the FM2 Ant to Gradle>
>> >> >> conversion, go ahead!>
>> >> >> >
>> >> >> >
>> >> >> Thursday, September 22, 2016, 1:08:18 PM, Mauricio Nu%uFFFDez
>> wrote:>
>> >> >> >
>> >> >>> Hi Daniel,>
>> >> >>> >
>> >> >>> Thanks for your feedback. I'm pushing to my personal github. I
>> have a>>
>> >> >>> parent pom with a submodule core and a submodule ext. I want to
>> check>>
>> >> >>> alternatives for the different artifacts. Maybe with reflection,
>> as>
>> >> >>> suggested by the previous effort, or retrotranslator.>
>> >> >>> >
>> >> >>> https://github.com/mauronunez/incubator-freemarker/tree/maven (
>> my fork,>>
>> >> >>> branch "maven" )>
>> >> >>> >
>> >> >>> I have more experience with maven, but I think this as
>> intermediate step>>
>> >> >>> for Gradle.>
>> >> >>> About maven for FM2, yes, I agree with to keep it as is, not
>> impacting>>
>> >> >>> users. I want to contribute my changes to FM3, but now, FM2 it's
>> a good>>
>> >> >>> codebase to test, and to be more involved as developer.>
>> >> >>> >
>> >> >>> Btw, I've create a pull request to fix some Junit tests.>
>> >> >>> >
>> >> >>> Regards,>
>> >> >>> >
>> >> >>> Mauricio>
>> >> >>> >
>> >> >>> 2016-09-21 14:49 GMT-03:00 Daniel Dekany <dd...@freemail.hu>:>
>> >> >>> >
>> >> >>>> Hi,>
>> >> >>>> >
>> >> >>>> In FreeMarker 3 we want to modularize the project, so that there
>> will>>
>> >> >>>> be at least a freemarker-core and then a separate artifacts for
>> each>>
>> >> >>>> extension (like freemarker-jsp, etc.). Thus so far I agree with
>> what>>
>> >> >>>> you are doing. If it can be done with FreeMarker 2 though, I'm
>> not>
>> >> >>>> sure, because we have to keep strict backward compatibility
>> there, and>>
>> >> >>>> also we have to consider how much hassle such a change causes
>> for the>>
>> >> >>>> users. (There are orders magnitude more users than contributors.>
>> >> >>>> Something that's nice for the contributors might bears no value
>> for>>
>> >> >>>> the users, or even just makes dependency management more fragile
>> for>>
>> >> >>>> them.)>
>> >> >>>> >
>> >> >>>> As of a monolithic Maven build (where the point is to migrate
>> away>
>> >> >>>> from Ant and make the build more "standard"), there was such
>> activity,>>
>> >> >>>> but it didn't get far enough so far:>
>> >> >>>> https://issues.apache.org/jira/browse/FREEMARKER-7?jql=>
>> >> >>>> project%20%3D%20FREEMARKER>
>> >> >>>> >
>> >> >>>> The main problem is that in FreeMarker it happens now and then
>> that we>>
>> >> >>>> have to depend on multiple versions of the same artifact (or on>
>> >> >>>> multiple versions of the Java platform). Ant can handle that, but>
>> >> >>>> Maven... well, perhaps if we extract parts that are above the
>> baseline>>
>> >> >>>> platform (Java 5 currently, but let's say Java 6) into separate>
>> >> >>>> artifacts, like freemarker-core-java-8 and such, but that won't
>> be>
>> >> >>>> backward compatible as users had to depend on them explicitly (I>
>> >> >>>> guess), and even if we consider FreeMarker 3, it might
>> complicates the>>
>> >> >>>> life of users. So perhaps we have to look into using Gradle.
>> (Gradle>>
>> >> >>>> can be also handy if we want to target Android, and with in
>> FreeMarker>>
>> >> >>>> 3 we surely should.)>
>> >> >>>> >
>> >> >>>> Also, there are quite a few extra steps and extra tasks, if you
>> look>>
>> >> >>>> into the build.xml. Though I guess most of them can be
>> reimplemented>>
>> >> >>>> in Maven gradually, and we can keep the Ant build until it's
>> done...>>
>> >> >>>> >
>> >> >>>> >
>> >> >>>> Tuesday, September 20, 2016, 10:03:56 PM, Mauricio Nu%uFFFDez
>> wrote:>>
>> >> >>>> >
>> >> >>>>> Hi all,>
>> >> >>>>> >
>> >> >>>>> I'm a older user of freemarker, from version 1. Today, I'm
>> working in a>>
>> >> >>>>> maven pom.xml for freemarker. Freemarker is very valuable for
>> me, and I>>
>> >> >>>>> want to contribute. My old web site ( www.chile.com ) is down
>> now, but>>
>> >> >>>> it>
>> >> >>>>> was developed with freemarker about 2002-2003.>
>> >> >>>>> >
>> >> >>>>> I'm using a maven plugin for javacc, and this build is working.
>> I have>>
>> >> >>>> some>
>> >> >>>>> issues with some JUnit tests, but I'm working in that ( I guess
>> the>>
>> >> >>>> problem>
>> >> >>>>> is related to my Locale )>
>> >> >>>>> >
>> >> >>>>> But my approach is to create a base freemarker project, and a
>> second>>
>> >> >>>>> project, related to extensions ( jython, servlet, etc ). Before>
>> >> >>>> continue, I>
>> >> >>>>> want some comments from you.>
>> >> >>>>> >
>> >> >>>>> Regards,>
>> >> >>>>> >
>> >> >>>>> Mauricio>
>> >> >>>> >
>> >> >>>> -->
>> >> >>>> Thanks,>
>> >> >>>> Daniel Dekany>
>> >> >>>> >
>> >> >>>> >
>> >> >> >
>> >> >> -- >
>> >> >> Thanks,>
>> >> >> Daniel Dekany>
>> >> >> >
>> >> >> >
>> >> >>
>> >> >>
>> >> >>
>> >>
>> >>
>> >
>>
>> --
>> Thanks,
>> Daniel Dekany
>>
>>
--
Thanks,
Daniel Dekany
