Re: Documenting Github PR jenkins trigger phrases

[Alan Myrvold]

This is tracked by https://issues.apache.org/jira/browse/BEAM-3068

On Fri, Jun 1, 2018 at 7:38 AM Ismaël Mejía  wrote:

> Any progress on this ? Is there a JIRA to track it ?
> On Fri, May 11, 2018 at 6:02 PM Jean-Baptiste Onofré 
> wrote:
> >
> > Agree, I thought there's already a PR about that.
> >
> > Regards
> > JB
> >
> > On 11/05/2018 16:11, Alexey Romanenko wrote:
> > > +1 to add such reference guide of Jenkins commands to Testing guide. It
> > > should be extremely useful, especially for those who were not aware
> > > about this before.
> > >
> > > WBR,
> > > Alexey
> > >
> > >> On 11 May 2018, at 02:44, Ankur Goenka  > >> > wrote:
> > >>
> > >> In my experience affect of white space in commit is inconsistent for
> > >> certain commands they do matter while for others they don't.
> > >>
> > >> On Thu, May 10, 2018 at 5:43 PM Valentyn Tymofieiev
> > >> mailto:valen...@google.com>> wrote:
> > >>
> > >> +1 to writing a Beam Jenkins spellbook.
> > >> I have observed that Jenkins commands sometimes don't work for the
> > >> first time, why could this be? Do end of lines at the end of
> > >> command matter?
> > >>
> > >>
> > >> On Thu, May 10, 2018 at 1:24 PM, Andrew Pilloud
> > >> mailto:apill...@google.com>> wrote:
> > >>
> > >> It would be great to have the set of "Run {Java,Python,Go}
> > >> PreCommit" documented in the contributors guide as well. Those
> > >> match up to the jobs auto run on every PR and are the ones I
> > >> use most. There is no security, anyone can run them including
> > >> 'Run Seed Job'. That one seems like a good one to document in
> > >> testing, because it is the one that loads changes to the rest.
> > >>
> > >> Andrew
> > >>
> > >> On Thu, May 10, 2018 at 12:27 PM Huygaa Batsaikhan
> > >> mailto:bat...@google.com>> wrote:
> > >>
> > >> Hi devs,
> > >>
> > >> We can run various jenkins commands (precommit,
> > >> postcommit, performance tests) directly from Github Pull
> > >> Request UI by commenting phrases such as "retest this
> > >> please". Unfortunately, this tool is not documented. I am
> > >> adding a brief documentation in
> > >> https://beam.apache.org/contribute/testing/ and I need
> > >> some help.
> > >>
> > >>  1. What are the most common phrases used?
> > >>  2. Can anyone run these commands? Are there any
> > >> permission issues?
> > >>  3. Does it make sense to categorize the commands as
> > >> Performance tests, Precommit, Postcommit, and Release
> > >> Validation?
> > >>
> > >> Let me know what you think,
> > >>
> > >> Thanks,
> > >> Huygaa
> > >>
> > >>
> > >
>

Re: Documenting Github PR jenkins trigger phrases

[Ismaël Mejía]

Any progress on this ? Is there a JIRA to track it ?
On Fri, May 11, 2018 at 6:02 PM Jean-Baptiste Onofré  wrote:
>
> Agree, I thought there's already a PR about that.
>
> Regards
> JB
>
> On 11/05/2018 16:11, Alexey Romanenko wrote:
> > +1 to add such reference guide of Jenkins commands to Testing guide. It
> > should be extremely useful, especially for those who were not aware
> > about this before.
> >
> > WBR,
> > Alexey
> >
> >> On 11 May 2018, at 02:44, Ankur Goenka  >> > wrote:
> >>
> >> In my experience affect of white space in commit is inconsistent for
> >> certain commands they do matter while for others they don't.
> >>
> >> On Thu, May 10, 2018 at 5:43 PM Valentyn Tymofieiev
> >> mailto:valen...@google.com>> wrote:
> >>
> >> +1 to writing a Beam Jenkins spellbook.
> >> I have observed that Jenkins commands sometimes don't work for the
> >> first time, why could this be? Do end of lines at the end of
> >> command matter?
> >>
> >>
> >> On Thu, May 10, 2018 at 1:24 PM, Andrew Pilloud
> >> mailto:apill...@google.com>> wrote:
> >>
> >> It would be great to have the set of "Run {Java,Python,Go}
> >> PreCommit" documented in the contributors guide as well. Those
> >> match up to the jobs auto run on every PR and are the ones I
> >> use most. There is no security, anyone can run them including
> >> 'Run Seed Job'. That one seems like a good one to document in
> >> testing, because it is the one that loads changes to the rest.
> >>
> >> Andrew
> >>
> >> On Thu, May 10, 2018 at 12:27 PM Huygaa Batsaikhan
> >> mailto:bat...@google.com>> wrote:
> >>
> >> Hi devs,
> >>
> >> We can run various jenkins commands (precommit,
> >> postcommit, performance tests) directly from Github Pull
> >> Request UI by commenting phrases such as "retest this
> >> please". Unfortunately, this tool is not documented. I am
> >> adding a brief documentation in
> >> https://beam.apache.org/contribute/testing/ and I need
> >> some help.
> >>
> >>  1. What are the most common phrases used?
> >>  2. Can anyone run these commands? Are there any
> >> permission issues?
> >>  3. Does it make sense to categorize the commands as
> >> Performance tests, Precommit, Postcommit, and Release
> >> Validation?
> >>
> >> Let me know what you think,
> >>
> >> Thanks,
> >> Huygaa
> >>
> >>
> >

Re: Documenting Github PR jenkins trigger phrases

[Jean-Baptiste Onofré]

Agree, I thought there's already a PR about that.

Regards
JB

On 11/05/2018 16:11, Alexey Romanenko wrote:
+1 to add such reference guide of Jenkins commands to Testing guide. It 
should be extremely useful, especially for those who were not aware 
about this before.


WBR,
Alexey

On 11 May 2018, at 02:44, Ankur Goenka > wrote:


In my experience affect of white space in commit is inconsistent for 
certain commands they do matter while for others they don't.


On Thu, May 10, 2018 at 5:43 PM Valentyn Tymofieiev 
> wrote:


+1 to writing a Beam Jenkins spellbook.
I have observed that Jenkins commands sometimes don't work for the
first time, why could this be? Do end of lines at the end of
command matter?


On Thu, May 10, 2018 at 1:24 PM, Andrew Pilloud
> wrote:

It would be great to have the set of "Run {Java,Python,Go}
PreCommit" documented in the contributors guide as well. Those
match up to the jobs auto run on every PR and are the ones I
use most. There is no security, anyone can run them including
'Run Seed Job'. That one seems like a good one to document in
testing, because it is the one that loads changes to the rest.

Andrew

On Thu, May 10, 2018 at 12:27 PM Huygaa Batsaikhan
> wrote:

Hi devs,

We can run various jenkins commands (precommit,
postcommit, performance tests) directly from Github Pull
Request UI by commenting phrases such as "retest this
please". Unfortunately, this tool is not documented. I am
adding a brief documentation in
https://beam.apache.org/contribute/testing/ and I need
some help.

 1. What are the most common phrases used?
 2. Can anyone run these commands? Are there any
permission issues?
 3. Does it make sense to categorize the commands as
Performance tests, Precommit, Postcommit, and Release
Validation?

Let me know what you think,

Thanks,
Huygaa

Re: Documenting Github PR jenkins trigger phrases

[Scott Wegner]

+1 to adding a doc for this, along with some PR conventions about when to
use these. Some questions that I would love to see documented:

* What tests are run automatically and when (pre-commits, all languages, on
every commit)
* What other test suites exist, and how should they be used (post-commits
and performance tests run on a schedule and validate already-merged code;
if a merge breaks post-commits we should prefer rollback over rollforward
to keep master in a healthy state; if you think your change could affect
one of these suites you can invoke them manually on your PR)
* How do reviewers use Jenkins signals (i.e. do reviewers expect
pre-commits to be green before reviewing PR's?)
* What should you do if you think a flaky test caused a failure on your PR?
(re-run tests using magical Jenkins incantation)

I also like the idea of automatically scraping the string from the
source-of-truth rather than manually maintaining. However, I suspect it
might be more trouble than it's worth (the groovy files are in a different
git repo than beam-site; the site is currently built from simple markdown
files which get compiled to HTML and checked-in). If sourcing the strings
from the source proves difficult, I'm in favor of going the simple route
and manually maintaining the list; I think there's sufficient value to
justify the maintenance cost of.

Another idea would be to see if we could query Jenkins via javascript on
page load for the set of beam jobs and trigger phrases. This could be a
simpler integration point and has the benefit of always being up-to-date
regardless of which commit was used to generate the jobs.

On Fri, May 11, 2018 at 7:11 AM Alexey Romanenko 
wrote:

> +1 to add such reference guide of Jenkins commands to Testing guide. It
> should be extremely useful, especially for those who were not aware about
> this before.
>
> WBR,
> Alexey
>
>
> On 11 May 2018, at 02:44, Ankur Goenka  wrote:
>
> In my experience affect of white space in commit is inconsistent for
> certain commands they do matter while for others they don't.
>
> On Thu, May 10, 2018 at 5:43 PM Valentyn Tymofieiev 
> wrote:
>
>> +1 to writing a Beam Jenkins spellbook.
>> I have observed that Jenkins commands sometimes don't work for the first
>> time, why could this be? Do end of lines at the end of command matter?
>>
>>
>> On Thu, May 10, 2018 at 1:24 PM, Andrew Pilloud 
>> wrote:
>>
>>> It would be great to have the set of "Run {Java,Python,Go} PreCommit"
>>> documented in the contributors guide as well. Those match up to the
>>> jobs auto run on every PR and are the ones I use most. There is no
>>> security, anyone can run them including 'Run Seed Job'. That one seems like
>>> a good one to document in testing, because it is the one that loads changes
>>> to the rest.
>>>
>>> Andrew
>>>
>>> On Thu, May 10, 2018 at 12:27 PM Huygaa Batsaikhan 
>>> wrote:
>>>
 Hi devs,

 We can run various jenkins commands (precommit, postcommit,
 performance tests) directly from Github Pull Request UI by commenting
 phrases such as "retest this please". Unfortunately, this tool is not
 documented. I am adding a brief documentation in
 https://beam.apache.org/contribute/testing/ and I need some help.

1. What are the most common phrases used?
2. Can anyone run these commands? Are there any permission issues?
3. Does it make sense to categorize the commands as Performance
tests, Precommit, Postcommit, and Release Validation?

 Let me know what you think,

 Thanks,
 Huygaa

>>>
>>
>

Re: Documenting Github PR jenkins trigger phrases

[Alexey Romanenko]

+1 to add such reference guide of Jenkins commands to Testing guide. It should 
be extremely useful, especially for those who were not aware about this before. 

WBR,
Alexey

> On 11 May 2018, at 02:44, Ankur Goenka  wrote:
> 
> In my experience affect of white space in commit is inconsistent for certain 
> commands they do matter while for others they don't.
> 
> On Thu, May 10, 2018 at 5:43 PM Valentyn Tymofieiev  > wrote:
> +1 to writing a Beam Jenkins spellbook. 
> I have observed that Jenkins commands sometimes don't work for the first 
> time, why could this be? Do end of lines at the end of command matter?
> 
> 
> On Thu, May 10, 2018 at 1:24 PM, Andrew Pilloud  > wrote:
> It would be great to have the set of "Run {Java,Python,Go} PreCommit" 
> documented in the contributors guide as well. Those match up to the jobs auto 
> run on every PR and are the ones I use most. There is no security, anyone can 
> run them including 'Run Seed Job'. That one seems like a good one to document 
> in testing, because it is the one that loads changes to the rest.
> 
> Andrew
> 
> On Thu, May 10, 2018 at 12:27 PM Huygaa Batsaikhan  > wrote:
> Hi devs,
> 
> We can run various jenkins commands (precommit, postcommit, performance 
> tests) directly from Github Pull Request UI by commenting phrases such as 
> "retest this please". Unfortunately, this tool is not documented. I am adding 
> a brief documentation in https://beam.apache.org/contribute/testing/ 
>  and I need some help.
> What are the most common phrases used?
> Can anyone run these commands? Are there any permission issues?
> Does it make sense to categorize the commands as Performance tests, 
> Precommit, Postcommit, and Release Validation?
> Let me know what you think,
> 
> Thanks,
> Huygaa
> 

Re: Documenting Github PR jenkins trigger phrases

[Ankur Goenka]

In my experience affect of white space in commit is inconsistent for
certain commands they do matter while for others they don't.

On Thu, May 10, 2018 at 5:43 PM Valentyn Tymofieiev 
wrote:

> +1 to writing a Beam Jenkins spellbook.
> I have observed that Jenkins commands sometimes don't work for the first
> time, why could this be? Do end of lines at the end of command matter?
>
>
> On Thu, May 10, 2018 at 1:24 PM, Andrew Pilloud 
> wrote:
>
>> It would be great to have the set of "Run {Java,Python,Go} PreCommit"
>> documented in the contributors guide as well. Those match up to the jobs auto
>> run on every PR and are the ones I use most. There is no security, anyone
>> can run them including 'Run Seed Job'. That one seems like a good one to
>> document in testing, because it is the one that loads changes to the rest.
>>
>> Andrew
>>
>> On Thu, May 10, 2018 at 12:27 PM Huygaa Batsaikhan 
>> wrote:
>>
>>> Hi devs,
>>>
>>> We can run various jenkins commands (precommit, postcommit, performance
>>> tests) directly from Github Pull Request UI by commenting phrases such
>>> as "retest this please". Unfortunately, this tool is not documented. I am
>>> adding a brief documentation in
>>> https://beam.apache.org/contribute/testing/ and I need some help.
>>>
>>>1. What are the most common phrases used?
>>>2. Can anyone run these commands? Are there any permission issues?
>>>3. Does it make sense to categorize the commands as Performance
>>>tests, Precommit, Postcommit, and Release Validation?
>>>
>>> Let me know what you think,
>>>
>>> Thanks,
>>> Huygaa
>>>
>>
>

Re: Documenting Github PR jenkins trigger phrases

[Valentyn Tymofieiev]

+1 to writing a Beam Jenkins spellbook.
I have observed that Jenkins commands sometimes don't work for the first
time, why could this be? Do end of lines at the end of command matter?


On Thu, May 10, 2018 at 1:24 PM, Andrew Pilloud  wrote:

> It would be great to have the set of "Run {Java,Python,Go} PreCommit"
> documented in the contributors guide as well. Those match up to the jobs auto
> run on every PR and are the ones I use most. There is no security, anyone
> can run them including 'Run Seed Job'. That one seems like a good one to
> document in testing, because it is the one that loads changes to the rest.
>
> Andrew
>
> On Thu, May 10, 2018 at 12:27 PM Huygaa Batsaikhan 
> wrote:
>
>> Hi devs,
>>
>> We can run various jenkins commands (precommit, postcommit, performance
>> tests) directly from Github Pull Request UI by commenting phrases such
>> as "retest this please". Unfortunately, this tool is not documented. I am
>> adding a brief documentation in https://beam.apache.org/
>> contribute/testing/ and I need some help.
>>
>>1. What are the most common phrases used?
>>2. Can anyone run these commands? Are there any permission issues?
>>3. Does it make sense to categorize the commands as Performance
>>tests, Precommit, Postcommit, and Release Validation?
>>
>> Let me know what you think,
>>
>> Thanks,
>> Huygaa
>>
>

Re: Documenting Github PR jenkins trigger phrases

[Andrew Pilloud]

It would be great to have the set of "Run {Java,Python,Go} PreCommit"
documented in the contributors guide as well. Those match up to the jobs auto
run on every PR and are the ones I use most. There is no security, anyone
can run them including 'Run Seed Job'. That one seems like a good one to
document in testing, because it is the one that loads changes to the rest.

Andrew

On Thu, May 10, 2018 at 12:27 PM Huygaa Batsaikhan 
wrote:

> Hi devs,
>
> We can run various jenkins commands (precommit, postcommit, performance
> tests) directly from Github Pull Request UI by commenting phrases such as
> "retest this please". Unfortunately, this tool is not documented. I am
> adding a brief documentation in
> https://beam.apache.org/contribute/testing/ and I need some help.
>
>1. What are the most common phrases used?
>2. Can anyone run these commands? Are there any permission issues?
>3. Does it make sense to categorize the commands as Performance tests,
>Precommit, Postcommit, and Release Validation?
>
> Let me know what you think,
>
> Thanks,
> Huygaa
>

Re: Documenting Github PR jenkins trigger phrases

[Jean-Baptiste Onofré]

That's a good idea. It could be helpful.

Regards
JB

On 05/10/2018 09:34 PM, Ankur Goenka wrote:
> These actions are documented in groovy scripts
> like 
> https://github.com/apache/beam/blob/b7bfca9b196699a096786506777a49237e0b3776/.test-infra/jenkins/job_beam_PostCommit_Python_ValidatesContainer_Dataflow.groovy#L36
> We can create a tool which can just go over these scripts and print the 
> comments
> eliminating the need to keep the document upto date.
> 
> 
> On Thu, May 10, 2018 at 12:33 PM Kenneth Knowles  > wrote:
> 
> It seems extremely useful to have a concise catalog of these commands in 
> the
> contribution guide. I think new contributors will perhaps not be reading 
> the
> testing guide but just want Jenkins to do the work for them.
> 
> Kenn
> 
> On Thu, May 10, 2018 at 12:27 PM Huygaa Batsaikhan  > wrote:
> 
> Hi devs,
> 
> We can run various jenkins commands (precommit, postcommit, 
> performance
> tests) directly from Github Pull Request UI by commenting phrases such
> as "retest this please". Unfortunately, this tool is not documented. I
> am adding a brief documentation in
> https://beam.apache.org/contribute/testing/ and I need some help.
> 
>  1. What are the most common phrases used?
>  2. Can anyone run these commands? Are there any permission issues?
>  3. Does it make sense to categorize the commands as Performance 
> tests,
> Precommit, Postcommit, and Release Validation?
> 
> Let me know what you think,
> 
> Thanks,
> Huygaa
> 

-- 
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Re: Documenting Github PR jenkins trigger phrases

[Kenneth Knowles]

It does seem useful to be able to automatically scrape the job definitions
to dump out the markdown to publish on the web site. It would probably make
the groovy definitions more readable to factor that out into some table of
constants with longer descriptions, anyhow.

Kenn

On Thu, May 10, 2018 at 12:35 PM Ankur Goenka  wrote:

> These actions are documented in groovy scripts like
> https://github.com/apache/beam/blob/b7bfca9b196699a096786506777a49237e0b3776/.test-infra/jenkins/job_beam_PostCommit_Python_ValidatesContainer_Dataflow.groovy#L36
> We can create a tool which can just go over these scripts and print the
> comments eliminating the need to keep the document upto date.
>
>
> On Thu, May 10, 2018 at 12:33 PM Kenneth Knowles  wrote:
>
>> It seems extremely useful to have a concise catalog of these commands in
>> the contribution guide. I think new contributors will perhaps not be
>> reading the testing guide but just want Jenkins to do the work for them.
>>
>> Kenn
>>
>> On Thu, May 10, 2018 at 12:27 PM Huygaa Batsaikhan 
>> wrote:
>>
>>> Hi devs,
>>>
>>> We can run various jenkins commands (precommit, postcommit, performance
>>> tests) directly from Github Pull Request UI by commenting phrases such
>>> as "retest this please". Unfortunately, this tool is not documented. I am
>>> adding a brief documentation in
>>> https://beam.apache.org/contribute/testing/ and I need some help.
>>>
>>>1. What are the most common phrases used?
>>>2. Can anyone run these commands? Are there any permission issues?
>>>3. Does it make sense to categorize the commands as Performance
>>>tests, Precommit, Postcommit, and Release Validation?
>>>
>>> Let me know what you think,
>>>
>>> Thanks,
>>> Huygaa
>>>
>>

Re: Documenting Github PR jenkins trigger phrases

[Ankur Goenka]

These actions are documented in groovy scripts like
https://github.com/apache/beam/blob/b7bfca9b196699a096786506777a49237e0b3776/.test-infra/jenkins/job_beam_PostCommit_Python_ValidatesContainer_Dataflow.groovy#L36
We can create a tool which can just go over these scripts and print the
comments eliminating the need to keep the document upto date.


On Thu, May 10, 2018 at 12:33 PM Kenneth Knowles  wrote:

> It seems extremely useful to have a concise catalog of these commands in
> the contribution guide. I think new contributors will perhaps not be
> reading the testing guide but just want Jenkins to do the work for them.
>
> Kenn
>
> On Thu, May 10, 2018 at 12:27 PM Huygaa Batsaikhan 
> wrote:
>
>> Hi devs,
>>
>> We can run various jenkins commands (precommit, postcommit, performance
>> tests) directly from Github Pull Request UI by commenting phrases such
>> as "retest this please". Unfortunately, this tool is not documented. I am
>> adding a brief documentation in
>> https://beam.apache.org/contribute/testing/ and I need some help.
>>
>>1. What are the most common phrases used?
>>2. Can anyone run these commands? Are there any permission issues?
>>3. Does it make sense to categorize the commands as Performance
>>tests, Precommit, Postcommit, and Release Validation?
>>
>> Let me know what you think,
>>
>> Thanks,
>> Huygaa
>>
>

Re: Documenting Github PR jenkins trigger phrases

[Kenneth Knowles]

It seems extremely useful to have a concise catalog of these commands in
the contribution guide. I think new contributors will perhaps not be
reading the testing guide but just want Jenkins to do the work for them.

Kenn

On Thu, May 10, 2018 at 12:27 PM Huygaa Batsaikhan 
wrote:

> Hi devs,
>
> We can run various jenkins commands (precommit, postcommit, performance
> tests) directly from Github Pull Request UI by commenting phrases such as
> "retest this please". Unfortunately, this tool is not documented. I am
> adding a brief documentation in
> https://beam.apache.org/contribute/testing/ and I need some help.
>
>1. What are the most common phrases used?
>2. Can anyone run these commands? Are there any permission issues?
>3. Does it make sense to categorize the commands as Performance tests,
>Precommit, Postcommit, and Release Validation?
>
> Let me know what you think,
>
> Thanks,
> Huygaa
>

Re: Documenting Github PR jenkins trigger phrases

[Jean-Baptiste Onofré]

Hi Huygaa,

Kenn already sent some days ago a mail about that.

AFAIR,  the contribution guide has been updated (or corresponding PR):

https://beam.apache.org/contribute/contribution-guide/

Regards
JB

On 05/10/2018 09:26 PM, Huygaa Batsaikhan wrote:
> Hi devs,
> 
> We can run various jenkins commands (precommit, postcommit, performance tests)
> directly from Github Pull Request UI by commenting phrases such as "retest 
> this
> please". Unfortunately, this tool is not documented. I am adding a brief
> documentation in https://beam.apache.org/contribute/testing/ and I need some 
> help.
> 
>  1. What are the most common phrases used?
>  2. Can anyone run these commands? Are there any permission issues?
>  3. Does it make sense to categorize the commands as Performance tests,
> Precommit, Postcommit, and Release Validation?
> 
> Let me know what you think,
> 
> Thanks,
> Huygaa

-- 
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Documenting Github PR jenkins trigger phrases

[Huygaa Batsaikhan]

Hi devs,

We can run various jenkins commands (precommit, postcommit, performance
tests) directly from Github Pull Request UI by commenting phrases such as
"retest this please". Unfortunately, this tool is not documented. I am
adding a brief documentation in https://beam.apache.org/contribute/testing/
and I need some help.

   1. What are the most common phrases used?
   2. Can anyone run these commands? Are there any permission issues?
   3. Does it make sense to categorize the commands as Performance tests,
   Precommit, Postcommit, and Release Validation?

Let me know what you think,

Thanks,
Huygaa