On Tue, Jan 23, 2018 at 2:51 PM, Romain Manni-Bucau <rmannibu...@gmail.com>
wrote:
Hmm, did you read it right Kenn? I think the idea was to skip all
> validation/sanity checks tasks at once (gradle xxxx -Pfast) instead of
> doing it manually (gradle -x findbugs -x checkstyle etc...)
>
Yes, I read it right. We all want the same thing - not doing a bunch of
extra useless unrequested stuff when developing. The concept of skipping is
backwards. We don't need configs that skip things, because in a correct
dependency-driven build they are already not running.
So since I don't want to pretend to know gradle's invocations yet I will
call it $TOOL. Here's a collection of imaginary commands:
$TOOL :sdks:java:core:unittest # or $TOOL test :sdks:java:core or
whatever
$TOOL :sdks:java:core:findbugs
$TOOL :sdks:java:core:checkstyle
$TOOL :sdks:java:core:javadoc
None of these causes any of the others to run. Anything else is a bug. The
`findbugs` and `test` cause a build of the needed jars and nothing else.
Another example:
$TOOL :runners:core-java:unittest
This builds the model, the core SDK, and the runners-core module, then runs
the unit tests of the runners-core module. It does not test SDK core, or
run any javadoc, findbugs, or checkstyle on any module. Anything else is a
bug.
Now, to build a precommit that is easy to reproduce on one line, you could
build a compound task
$TOOL :sdks:java:core:precommit # runs a selection of targets that we
define
At this point you might want to skip things from the :verify task here. But
really, you probably just want to run the things you are interested in and
you don't need custom hooks in the aggregated task.
My understanding is that gradle can support all of this, if we are
disciplined. Getting to this point is the main/only reason I supported
gradle.
Kenn
>
>
>
>> Kenn
>>
>>
>>
>>> >>
>>> >> diff --git a/examples/java/build.gradle b/examples/java/build.gradle
>>> >> index 0fc0b17df..001bd8b38 100644
>>> >> --- a/examples/java/build.gradle
>>> >> +++ b/examples/java/build.gradle
>>> >> @@ -130,7 +130,7 @@ def preCommitAdditionalFlags = [
>>> >> dataflowStreamingRunner: [ "--streaming=true" ],
>>> >> ]
>>> >> for (String runner : preCommitRunners) {
>>> >> - tasks.create(name: runner + "PreCommit", type: Test) {
>>> >> + tasks.create(name: runner + "PreCommit", type: Test, description:
>>> "Run tests
>>> >> for runner ${runner.replace('Runner', '')}") {
>>> >> def preCommitBeamTestPipelineOptions = [
>>> >> "--project=apache-beam-testing",
>>> >> "--tempRoot=gs://temp-storage-for-end-to-end-tests",
>>> >>
>>> >>
>>> >>
>>> >> Romain Manni-Bucau
>>> >> @rmannibucau <https://twitter.com/rmannibucau> | Blog
>>> >> <https://rmannibucau.metawerx.net/> | Old Blog
>>> >> <http://rmannibucau.wordpress.com> | Github <
>>> https://github.com/rmannibucau> |
>>> >> LinkedIn <https://www.linkedin.com/in/rmannibucau>
>>> >>
>>> >> 2018-01-23 8:45 GMT+01:00 Jean-Baptiste Onofré <j...@nanthrax.net
>>> >> <mailto:j...@nanthrax.net>>:
>>> >>
>>> >> Hi Romain,
>>> >>
>>> >> I think we are pretty close: agree to add some explicit tasks &
>>> projects names.
>>> >>
>>> >> We can add additional tasks like skipAudit, for instance.
>>> >>
>>> >> As reminder, gradle tasks provides the list of tasks and gradle
>>> projects
>>> >> provides the list of projects/modules.
>>> >>
>>> >> Regards
>>> >> JB
>>> >>
>>> >> On 01/23/2018 08:12 AM, Romain Manni-Bucau wrote:
>>> >> > Hmm, I have to admit docs dont have my favor cause they are
>>> easily outdated and
>>> >> > hard to search but you hit a good point. Starting by renaming
>>> properly the tasks
>>> >> > and maybe writing what is done in build files - since it is
>>> code and even "api
>>> >> > for dev", it requires as much comments than the main api - can
>>> be better to start.
>>> >> >
>>> >> > Also a big switch flag to bypass checkstyle/findbugs/... can be
>>> good while in
>>> >> > dev since these phases cost a looot for nothing while you
>>> validates your code in
>>> >> > runners modules for instance.
>>> >> >
>>> >> > Le 23 janv. 2018 07:15, "Kenneth Knowles" <k...@google.com
>>> <mailto:k...@google.com>
>>> >> > <mailto:k...@google.com <mailto:k...@google.com>>> a écrit :
>>> >> >
>>> >> > On Mon, Jan 22, 2018 at 10:03 PM, Romain Manni-Bucau <
>>> rmannibu...@gmail.com <mailto:rmannibu...@gmail.com>
>>> >> > <mailto:rmannibu...@gmail.com <mailto:rmannibu...@gmail.com
>>> >>> wrote:
>>> >> >
>>> >> > @Kenneth: why not dropping the doc for a script with
>>> comments in the
>>> >> > project? A "RUNME.sh" ;).
>>> >> >
>>> >> >
>>> >> > That's cool, too, but also any shell one liner can be a
>>> gradle one
>>> >> liner or
>>> >> > mvn two/three liner :-). it is just trading one command
>>> that you cannot
>>> >> > guess easily for a different one that you still can't guess
>>> easily.
>>> >> >
>>> >> > For example, are the SparkRunner ValidatesRunner tests in
>>> the
>>> >> SparkRunner or
>>> >> > the core SDK or a third module that integrates the two? And
>>> why would you
>>> >> > know that the example ITs are called
>>> "sparkRunnerPreCommit"? It
>>> >> doesn't even
>>> >> > make sense really to have "precommit" or "postcommit"
>>> except as aliases to
>>> >> > make it easy to repro Jenkins' behavior - they have no
>>> other intrinsic
>>> >> meaning.
>>> >> >
>>> >> > So I was proposing a mapping from "full sentence +
>>> description" to one
>>> >> liner
>>> >> > to help people navigate the targets that we set up. Some
>>> web page or doc
>>> >> > that people can just quickly scan to find out to do common
>>> things, easier
>>> >> > than groovy or XML.
>>> >> >
>>> >> > Kenn
>>> >> >
>>> >> >
>>> >>
>>> >> --
>>> >> Jean-Baptiste Onofré
>>> >> jbono...@apache.org <mailto:jbono...@apache.org>
>>> >> http://blog.nanthrax.net
>>> >> Talend - http://www.talend.com
>>> >>
>>> >>
>>> >
>>> > --
>>> > Jean-Baptiste Onofré
>>> > jbono...@apache.org
>>> > http://blog.nanthrax.net
>>> > Talend - http://www.talend.com
>>>
>>
>>
>
