[
https://issues.apache.org/jira/browse/BEAM-4512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16517299#comment-16517299
]
Kenneth Knowles commented on BEAM-4512:
---------------------------------------
I took a quick look at this but eventually gave up due to running out of time.
Here is a dump of what I learned.
Context:
- we want to build using a hermetic offline clone of the dependencies
- we want to do it completely noninvasively, leaving the codebase unchanged
These are general needs that make sense in general, so I think it is OK to
adjust the Beam codebase in _general_ ways to support it.
Learnings:
- the recommended way to inject repositories is with {{--init-script}} but it
doesn't seem to work; there are many factors
- setting up {{repositories}} is fine, but we need to set up
{{buildscript.repositories}} aka plugin repositories, which works much less well
- {{buildscript}} sections are spec'd to only apply to the current file, so
having this in the init script does not help (see
[https://github.com/gradle/gradle/issues/1323] and
[https://discuss.gradle.org/t/init-scripts-classpath-dependencies-are-not-accessible-in-the-build-script/7044/5])
- also by the above {{build_rules.gradle}} does not properly declare its
dependencies, but _accidentally_ gets the right deps from {{build.gradle}}
- {{pluginManagement}} is a way to set up plugin repositories in an init script
- {{pluginManagement}} only works if you use the {{plugins}} block, but we use
the legacy/deprecated way of declaring {{classpath}} dependencies in a
{{buildscript}} block
- {{plugins}} block is only applicable to {{build.gradle}}, since an included
file is a "script plugin", not a build script
- we have dependency skew between {{build_rules.gradle}} shadow plugin and
{{build.gradle}} shadow plugin; this is benign
- putting a plugin in {{buildSrc}} allows simply declaring dependencies and
repositories like normal in {{buildSrc/build.gradle}}, so that is nice
- but init script also does not affect buildSrc (will fix, but unknown
timeframe: [https://github.com/gradle/gradle/issues/1055]) so we need to
hand-roll some way of injecting the repository
- init script can be used in a standalone plugin build, which our main build
can then use; standalone is not that different from {{buildSrc}}, which is also
not that different (actually much more readable/testable in the long run) than
a script plugin {{build_rules.gradle}}
- with a major refactor to put the "natures" on "ext" via the root
{{build.gradle}} we may be able to get the init script to work most easily, but
this has undesirable lock-in effects versus a plugin
So all-in-all my recommendation is to finish up the move to {{buildSrc}} and
then to modify {{buildSrc/build.gradle}} to allow injecting repositories via
{{-P}} hacks. From there, the Beam project and codebase has done plenty to make
it possible to build in a custom environment.
> Move DataflowRunner off of Maven build files
> --------------------------------------------
>
> Key: BEAM-4512
> URL: https://issues.apache.org/jira/browse/BEAM-4512
> Project: Beam
> Issue Type: Sub-task
> Components: runner-dataflow
> Reporter: Chamikara Jayalath
> Assignee: Mark Liu
> Priority: Major
>
> Currently DataflowRunner (internally at Google) depends on Beam's Maven build
> files. We have to move some internal build targets to use Gradle so that
> Maven files can be deleted from Beam.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
