Romain - really good point about docs getting out of date.
> On 01/23/2018 09:29 AM, Romain Manni-Bucau wrote:
> >> Yep,
> >>
> >> a compromise can be to ensure all custom tasks have a description maybe:
>
This is a great idea.
Let's do both! I think the thing that the comments in the gradle code
cannot capture are the ways that you might combine them, like the way you
override properties, etc.
On Tue, Jan 23, 2018 at 9:13 AM, Ismaël Mejía <ieme...@gmail.com> wrote:
> I used to test kafka
> with a different version of the client quickly by doing this with
> maven:
>
> mvn verify -Prelease -Dkafka.clients.version=0.10.0.1 -pl
> 'sdks/java/io/kafka'
>
> but I don't have any idea of how to do this on Gradle.
>
Did you figure this out. Luke - can you suggest something?
Of course everybody has a different workflow, but I am pretty sure
> there are some common tasks that we can document for people (like me)
> that are new to gradle.
Yea, actually, it never even occurred to me that you would use the command
line to test against other Kafka versions :-).
I have private gists with lots of maven invocations for doing things like
running ValidatesRunner or example ITs on just a particular runner. It
always requires multiple maven commands that are both multiple lines. I was
halfway for a while but I am now all gradle. I will start building the same
thing.
> - How to skip the Java/Python/Go build depending on your priorities.
>
IMO they should NOT run by default. Everything should be dependency-driven.
When I ask to run Java SDK tests, or Java examples IT on the Flink Runner,
it is *incorrect* for any Python or Go builds to run.
> - How to run an individual IntegrationTest or ValidatesRunner test.
>
Yes, #1 use case
> - How to skip findbugs,checkstyle,javadoc gneeration, etc. To have an
> ultra quick build.
>
Again, I think they should be independent tasks. I should be able to run
*any* of them without running *any* of the others. It is incorrect for
anything to cause any other thing to run if it does not directly require
its outputs.
There can be an aggregated "verify" command but I will actually very rarely
run that until I am done with a large chunk of work.
As for what the aggregated "verify" command does, people kept arguing about
what to make default. As long as we have a correct build for individual
checks (aka not running extra things) then I am happy for the default to be
long and slow, but we should still build profiles for both.
Kenn
> >>
> >> diff --git a/examples/java/build.gradle b/examples/java/build.gradle
> >> index 0fc0b17df..001bd8b38 100644
> >> --- a/examples/java/build.gradle
> >> +++ b/examples/java/build.gradle
> >> @@ -130,7 +130,7 @@ def preCommitAdditionalFlags = [
> >> dataflowStreamingRunner: [ "--streaming=true" ],
> >> ]
> >> for (String runner : preCommitRunners) {
> >> - tasks.create(name: runner + "PreCommit", type: Test) {
> >> + tasks.create(name: runner + "PreCommit", type: Test, description:
> "Run tests
> >> for runner ${runner.replace('Runner', '')}") {
> >> def preCommitBeamTestPipelineOptions = [
> >> "--project=apache-beam-testing",
> >> "--tempRoot=gs://temp-storage-for-end-to-end-tests",
> >>
> >>
> >>
> >> Romain Manni-Bucau
> >> @rmannibucau <https://twitter.com/rmannibucau> | Blog
> >> <https://rmannibucau.metawerx.net/> | Old Blog
> >> <http://rmannibucau.wordpress.com> | Github <https://github.com/
> rmannibucau> |
> >> LinkedIn <https://www.linkedin.com/in/rmannibucau>
> >>
> >> 2018-01-23 8:45 GMT+01:00 Jean-Baptiste Onofré <j...@nanthrax.net
> >> <mailto:j...@nanthrax.net>>:
> >>
> >> Hi Romain,
> >>
> >> I think we are pretty close: agree to add some explicit tasks &
> projects names.
> >>
> >> We can add additional tasks like skipAudit, for instance.
> >>
> >> As reminder, gradle tasks provides the list of tasks and gradle
> projects
> >> provides the list of projects/modules.
> >>
> >> Regards
> >> JB
> >>
> >> On 01/23/2018 08:12 AM, Romain Manni-Bucau wrote:
> >> > Hmm, I have to admit docs dont have my favor cause they are
> easily outdated and
> >> > hard to search but you hit a good point. Starting by renaming
> properly the tasks
> >> > and maybe writing what is done in build files - since it is code
> and even "api
> >> > for dev", it requires as much comments than the main api - can be
> better to start.
> >> >
> >> > Also a big switch flag to bypass checkstyle/findbugs/... can be
> good while in
> >> > dev since these phases cost a looot for nothing while you
> validates your code in
> >> > runners modules for instance.
> >> >
> >> > Le 23 janv. 2018 07:15, "Kenneth Knowles" <k...@google.com
> <mailto:k...@google.com>
> >> > <mailto:k...@google.com <mailto:k...@google.com>>> a écrit :
> >> >
> >> > On Mon, Jan 22, 2018 at 10:03 PM, Romain Manni-Bucau <
> rmannibu...@gmail.com <mailto:rmannibu...@gmail.com>
> >> > <mailto:rmannibu...@gmail.com <mailto:rmannibu...@gmail.com>>>
> wrote:
> >> >
> >> > @Kenneth: why not dropping the doc for a script with
> comments in the
> >> > project? A "RUNME.sh" ;).
> >> >
> >> >
> >> > That's cool, too, but also any shell one liner can be a
> gradle one
> >> liner or
> >> > mvn two/three liner :-). it is just trading one command that
> you cannot
> >> > guess easily for a different one that you still can't guess
> easily.
> >> >
> >> > For example, are the SparkRunner ValidatesRunner tests in the
> >> SparkRunner or
> >> > the core SDK or a third module that integrates the two? And
> why would you
> >> > know that the example ITs are called "sparkRunnerPreCommit"?
> It
> >> doesn't even
> >> > make sense really to have "precommit" or "postcommit" except
> as aliases to
> >> > make it easy to repro Jenkins' behavior - they have no other
> intrinsic
> >> meaning.
> >> >
> >> > So I was proposing a mapping from "full sentence +
> description" to one
> >> liner
> >> > to help people navigate the targets that we set up. Some web
> page or doc
> >> > that people can just quickly scan to find out to do common
> things, easier
> >> > than groovy or XML.
> >> >
> >> > Kenn
> >> >
> >> >
> >>
> >> --
> >> Jean-Baptiste Onofré
> >> jbono...@apache.org <mailto:jbono...@apache.org>
> >> http://blog.nanthrax.net
> >> Talend - http://www.talend.com
> >>
> >>
> >
> > --
> > Jean-Baptiste Onofré
> > jbono...@apache.org
> > http://blog.nanthrax.net
> > Talend - http://www.talend.com
>
