Re: New Committers - community

[Enrico Olivelli]

Il gio 3 gen 2019, 22:37 Tibor Digana  ha scritto:

> Why the build cannot be triggered on GitHub or PR?
> It's only a URL.
>

You are right, github gives special refs which represents the branch of the
forked repo, like pull/123/head.
The  problem is that our CI is configured for gitbox and not for github, so
we don't have those special refs

Enrico

If you see the frequency of PRs in Surefire, 1 or 2 PRs/month, this should
> not be a problem since the Windows build takes 1.5 hour and Linux 1 hour.
> Taking JDK 7 and 11, and one Maven version (3.5) is usually enough on Git
> branches and should be fine for PRs as well.
>
> On Thu, Jan 3, 2019 at 10:19 PM Stephen Connolly <
> stephen.alan.conno...@gmail.com> wrote:
>
> > On Thu 3 Jan 2019 at 20:52, Tibor Digana  wrote:
> >
> > > I already lost the point of this thread.
> > > Still the disk space problem on Windows executors in ASF Jenkins?
> > > If it's this issue, then why the workspace is not investigated directly
> > on
> > > the system with remote access?
> > > Jenkins is very good tool and I do not want to use Travis but Travis is
> > one
> > > step after finding the root cause.
> >
> >
> > We’re not going to be able to use Travis for Surefire (unless we have a
> > “fast” suite of tests)
> >
> > That doesn’t mean we cannot use Travis for the 90 other repos we have in
> > order to get test results of PRs from non-committers.
> >
> >
> > > T
> > >
> > > On Thu, Jan 3, 2019 at 8:39 PM Stephen Connolly <
> > > stephen.alan.conno...@gmail.com> wrote:
> > >
> > > > On Thu 3 Jan 2019 at 18:03, Enrico Olivelli 
> > wrote:
> > > >
> > > > > Il gio 3 gen 2019, 17:38 Mickael Istria  ha
> > > scritto:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I think this discussion is diverging into "trying TravisCI for
> some
> > > > > > plugins" and is loosing focus on the initial question of how to
> > > improve
> > > > > the
> > > > > > build+test flow to get faster feedback as a contributor or as a
> > > > reviewer.
> > > >
> > > >
> > > > Travis will get the build+test flow faster for non-committers.
> > > >
> > > >
> > > > > > Can the GitHub PR builder plugin be enabled on Maven Core ?
> > > >
> > > >
> > > > No, it’s not compatible with how we build, as currently we only build
> > > from
> > > > gitbox not from GitHub so there is no link for Jenkins to see
> > > >
> > > >
> > > > Committers
> > > > > > would be allowed to trigger a test with a single comment once
> they
> > > > > checked
> > > > > > it doesn't cause a security flaw.
> > > > > >
> > > > >
> > > > > It turns out that it is not simple as it could seem because we have
> > > > custom
> > > > > Jenkins plugins to scan all the repos and have a common
> > configuration,
> > > so
> > > > > in order to achieve such goal we need to enhance that plugin,
> please
> > > > > Stephen correct me if I am wrong.
> > > >
> > > >
> > > > Yep I need to enhance the plugin a little... just trying to clear
> stuff
> > > off
> > > > my plate
> > > >
> > > >
> > > > > Travis jumped on this train because it is easy to enable and it is
> > very
> > > > > widespread, and it leaves security problems out of ASF infra.
> > > > > But a check with Travis won't ever cover all jobs we are actually
> > > > starting
> > > > > per-branch.
> > > > > It can be a good compromise to start, but if we have resources to
> > > improve
> > > > > current integration this will be the best choice for the mid term.
> > > > >
> > > > > Enrico
> > > > >
> > > > > >
> > > > > > Cheers,
> > > > > >
> > > > > --
> > > > >
> > > > >
> > > > > -- Enrico Olivelli
> > > > >
> > > > --
> > > > Sent from my phone
> > > >
> > >
> > --
> > Sent from my phone
> >
>
-- 


-- Enrico Olivelli

Re: New Committers - community

[Tibor Digana]

I am doing #2.
This means I create a GitBox branch or I run it on my own PC. Not a problem
at all because usually the contributor does not complete it and I have to
add some test or clarify the change in documentation. Sometime I have to
rework the PR even if the idea was good, and I have to squash the commits.
So I am quite patient while triggering the CI via a branch but I cannot
represent the colleagues of course. Running PRs automatically would be
really a good idea.
T

On Thu, Jan 3, 2019 at 10:45 PM Stephen Connolly <
stephen.alan.conno...@gmail.com> wrote:

> On Thu 3 Jan 2019 at 21:37, Tibor Digana  wrote:
>
> > Why the build cannot be triggered on GitHub or PR?
> > It's only a URL.
> > If you see the frequency of PRs in Surefire, 1 or 2 PRs/month, this
> should
> > not be a problem since the Windows build takes 1.5 hour and Linux 1 hour.
> > Taking JDK 7 and 11, and one Maven version (3.5) is usually enough on Git
> > branches and should be fine for PRs as well.
>
>
> 1. The current branch source only discovers branches in gitbox, it doesn’t
> discover PRs on GitHub.
>
> 2. PRs on GitHub can have “untrusted” code, which shouldn’t be run within
> the ASF hardware without review by a committer.
>
> #1 can be solved by me enhancing the Jenkins plugin we use
>
> #2 either means asking committers to trigger builds after reviewing (which
> is a manual step and manual steps get forgotten)
>
> So instead we get Travis to do a first round CI for the PRs on GitHub. Then
> we can still have #1 and #2 as final pre-merge confirmation... but Travis
> gives the contributor immediate feedback on their contribution and that
> helps grow the community.
>
>
> >
> > On Thu, Jan 3, 2019 at 10:19 PM Stephen Connolly <
> > stephen.alan.conno...@gmail.com> wrote:
> >
> > > On Thu 3 Jan 2019 at 20:52, Tibor Digana 
> wrote:
> > >
> > > > I already lost the point of this thread.
> > > > Still the disk space problem on Windows executors in ASF Jenkins?
> > > > If it's this issue, then why the workspace is not investigated
> directly
> > > on
> > > > the system with remote access?
> > > > Jenkins is very good tool and I do not want to use Travis but Travis
> is
> > > one
> > > > step after finding the root cause.
> > >
> > >
> > > We’re not going to be able to use Travis for Surefire (unless we have a
> > > “fast” suite of tests)
> > >
> > > That doesn’t mean we cannot use Travis for the 90 other repos we have
> in
> > > order to get test results of PRs from non-committers.
> > >
> > >
> > > > T
> > > >
> > > > On Thu, Jan 3, 2019 at 8:39 PM Stephen Connolly <
> > > > stephen.alan.conno...@gmail.com> wrote:
> > > >
> > > > > On Thu 3 Jan 2019 at 18:03, Enrico Olivelli 
> > > wrote:
> > > > >
> > > > > > Il gio 3 gen 2019, 17:38 Mickael Istria  ha
> > > > scritto:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I think this discussion is diverging into "trying TravisCI for
> > some
> > > > > > > plugins" and is loosing focus on the initial question of how to
> > > > improve
> > > > > > the
> > > > > > > build+test flow to get faster feedback as a contributor or as a
> > > > > reviewer.
> > > > >
> > > > >
> > > > > Travis will get the build+test flow faster for non-committers.
> > > > >
> > > > >
> > > > > > > Can the GitHub PR builder plugin be enabled on Maven Core ?
> > > > >
> > > > >
> > > > > No, it’s not compatible with how we build, as currently we only
> build
> > > > from
> > > > > gitbox not from GitHub so there is no link for Jenkins to see
> > > > >
> > > > >
> > > > > Committers
> > > > > > > would be allowed to trigger a test with a single comment once
> > they
> > > > > > checked
> > > > > > > it doesn't cause a security flaw.
> > > > > > >
> > > > > >
> > > > > > It turns out that it is not simple as it could seem because we
> have
> > > > > custom
> > > > > > Jenkins plugins to scan all the repos and have a common
> > > configuration,
> > > > so
> > > > > > in order to achieve such goal we need to enhance that plugin,
> > please
> > > > > > Stephen correct me if I am wrong.
> > > > >
> > > > >
> > > > > Yep I need to enhance the plugin a little... just trying to clear
> > stuff
> > > > off
> > > > > my plate
> > > > >
> > > > >
> > > > > > Travis jumped on this train because it is easy to enable and it
> is
> > > very
> > > > > > widespread, and it leaves security problems out of ASF infra.
> > > > > > But a check with Travis won't ever cover all jobs we are actually
> > > > > starting
> > > > > > per-branch.
> > > > > > It can be a good compromise to start, but if we have resources to
> > > > improve
> > > > > > current integration this will be the best choice for the mid
> term.
> > > > > >
> > > > > > Enrico
> > > > > >
> > > > > > >
> > > > > > > Cheers,
> > > > > > >
> > > > > > --
> > > > > >
> > > > > >
> > > > > > -- Enrico Olivelli
> > > > > >
> > > > > --
> > > > > Sent from my phone
> > > > >
> > > >
> > > --
> > > Sent from my phone
> > >
> >
> --
> Sent from my phone
>

Re: New Committers - community

[Stephen Connolly]

On Thu 3 Jan 2019 at 21:37, Tibor Digana  wrote:

> Why the build cannot be triggered on GitHub or PR?
> It's only a URL.
> If you see the frequency of PRs in Surefire, 1 or 2 PRs/month, this should
> not be a problem since the Windows build takes 1.5 hour and Linux 1 hour.
> Taking JDK 7 and 11, and one Maven version (3.5) is usually enough on Git
> branches and should be fine for PRs as well.


1. The current branch source only discovers branches in gitbox, it doesn’t
discover PRs on GitHub.

2. PRs on GitHub can have “untrusted” code, which shouldn’t be run within
the ASF hardware without review by a committer.

#1 can be solved by me enhancing the Jenkins plugin we use

#2 either means asking committers to trigger builds after reviewing (which
is a manual step and manual steps get forgotten)

So instead we get Travis to do a first round CI for the PRs on GitHub. Then
we can still have #1 and #2 as final pre-merge confirmation... but Travis
gives the contributor immediate feedback on their contribution and that
helps grow the community.


>
> On Thu, Jan 3, 2019 at 10:19 PM Stephen Connolly <
> stephen.alan.conno...@gmail.com> wrote:
>
> > On Thu 3 Jan 2019 at 20:52, Tibor Digana  wrote:
> >
> > > I already lost the point of this thread.
> > > Still the disk space problem on Windows executors in ASF Jenkins?
> > > If it's this issue, then why the workspace is not investigated directly
> > on
> > > the system with remote access?
> > > Jenkins is very good tool and I do not want to use Travis but Travis is
> > one
> > > step after finding the root cause.
> >
> >
> > We’re not going to be able to use Travis for Surefire (unless we have a
> > “fast” suite of tests)
> >
> > That doesn’t mean we cannot use Travis for the 90 other repos we have in
> > order to get test results of PRs from non-committers.
> >
> >
> > > T
> > >
> > > On Thu, Jan 3, 2019 at 8:39 PM Stephen Connolly <
> > > stephen.alan.conno...@gmail.com> wrote:
> > >
> > > > On Thu 3 Jan 2019 at 18:03, Enrico Olivelli 
> > wrote:
> > > >
> > > > > Il gio 3 gen 2019, 17:38 Mickael Istria  ha
> > > scritto:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I think this discussion is diverging into "trying TravisCI for
> some
> > > > > > plugins" and is loosing focus on the initial question of how to
> > > improve
> > > > > the
> > > > > > build+test flow to get faster feedback as a contributor or as a
> > > > reviewer.
> > > >
> > > >
> > > > Travis will get the build+test flow faster for non-committers.
> > > >
> > > >
> > > > > > Can the GitHub PR builder plugin be enabled on Maven Core ?
> > > >
> > > >
> > > > No, it’s not compatible with how we build, as currently we only build
> > > from
> > > > gitbox not from GitHub so there is no link for Jenkins to see
> > > >
> > > >
> > > > Committers
> > > > > > would be allowed to trigger a test with a single comment once
> they
> > > > > checked
> > > > > > it doesn't cause a security flaw.
> > > > > >
> > > > >
> > > > > It turns out that it is not simple as it could seem because we have
> > > > custom
> > > > > Jenkins plugins to scan all the repos and have a common
> > configuration,
> > > so
> > > > > in order to achieve such goal we need to enhance that plugin,
> please
> > > > > Stephen correct me if I am wrong.
> > > >
> > > >
> > > > Yep I need to enhance the plugin a little... just trying to clear
> stuff
> > > off
> > > > my plate
> > > >
> > > >
> > > > > Travis jumped on this train because it is easy to enable and it is
> > very
> > > > > widespread, and it leaves security problems out of ASF infra.
> > > > > But a check with Travis won't ever cover all jobs we are actually
> > > > starting
> > > > > per-branch.
> > > > > It can be a good compromise to start, but if we have resources to
> > > improve
> > > > > current integration this will be the best choice for the mid term.
> > > > >
> > > > > Enrico
> > > > >
> > > > > >
> > > > > > Cheers,
> > > > > >
> > > > > --
> > > > >
> > > > >
> > > > > -- Enrico Olivelli
> > > > >
> > > > --
> > > > Sent from my phone
> > > >
> > >
> > --
> > Sent from my phone
> >
>
-- 
Sent from my phone

Re: New Committers - community

[Tibor Digana]

Why the build cannot be triggered on GitHub or PR?
It's only a URL.
If you see the frequency of PRs in Surefire, 1 or 2 PRs/month, this should
not be a problem since the Windows build takes 1.5 hour and Linux 1 hour.
Taking JDK 7 and 11, and one Maven version (3.5) is usually enough on Git
branches and should be fine for PRs as well.

On Thu, Jan 3, 2019 at 10:19 PM Stephen Connolly <
stephen.alan.conno...@gmail.com> wrote:

> On Thu 3 Jan 2019 at 20:52, Tibor Digana  wrote:
>
> > I already lost the point of this thread.
> > Still the disk space problem on Windows executors in ASF Jenkins?
> > If it's this issue, then why the workspace is not investigated directly
> on
> > the system with remote access?
> > Jenkins is very good tool and I do not want to use Travis but Travis is
> one
> > step after finding the root cause.
>
>
> We’re not going to be able to use Travis for Surefire (unless we have a
> “fast” suite of tests)
>
> That doesn’t mean we cannot use Travis for the 90 other repos we have in
> order to get test results of PRs from non-committers.
>
>
> > T
> >
> > On Thu, Jan 3, 2019 at 8:39 PM Stephen Connolly <
> > stephen.alan.conno...@gmail.com> wrote:
> >
> > > On Thu 3 Jan 2019 at 18:03, Enrico Olivelli 
> wrote:
> > >
> > > > Il gio 3 gen 2019, 17:38 Mickael Istria  ha
> > scritto:
> > > >
> > > > > Hi,
> > > > >
> > > > > I think this discussion is diverging into "trying TravisCI for some
> > > > > plugins" and is loosing focus on the initial question of how to
> > improve
> > > > the
> > > > > build+test flow to get faster feedback as a contributor or as a
> > > reviewer.
> > >
> > >
> > > Travis will get the build+test flow faster for non-committers.
> > >
> > >
> > > > > Can the GitHub PR builder plugin be enabled on Maven Core ?
> > >
> > >
> > > No, it’s not compatible with how we build, as currently we only build
> > from
> > > gitbox not from GitHub so there is no link for Jenkins to see
> > >
> > >
> > > Committers
> > > > > would be allowed to trigger a test with a single comment once they
> > > > checked
> > > > > it doesn't cause a security flaw.
> > > > >
> > > >
> > > > It turns out that it is not simple as it could seem because we have
> > > custom
> > > > Jenkins plugins to scan all the repos and have a common
> configuration,
> > so
> > > > in order to achieve such goal we need to enhance that plugin, please
> > > > Stephen correct me if I am wrong.
> > >
> > >
> > > Yep I need to enhance the plugin a little... just trying to clear stuff
> > off
> > > my plate
> > >
> > >
> > > > Travis jumped on this train because it is easy to enable and it is
> very
> > > > widespread, and it leaves security problems out of ASF infra.
> > > > But a check with Travis won't ever cover all jobs we are actually
> > > starting
> > > > per-branch.
> > > > It can be a good compromise to start, but if we have resources to
> > improve
> > > > current integration this will be the best choice for the mid term.
> > > >
> > > > Enrico
> > > >
> > > > >
> > > > > Cheers,
> > > > >
> > > > --
> > > >
> > > >
> > > > -- Enrico Olivelli
> > > >
> > > --
> > > Sent from my phone
> > >
> >
> --
> Sent from my phone
>

Re: New Committers - community

[Stephen Connolly]

On Thu 3 Jan 2019 at 20:52, Tibor Digana  wrote:

> I already lost the point of this thread.
> Still the disk space problem on Windows executors in ASF Jenkins?
> If it's this issue, then why the workspace is not investigated directly on
> the system with remote access?
> Jenkins is very good tool and I do not want to use Travis but Travis is one
> step after finding the root cause.


We’re not going to be able to use Travis for Surefire (unless we have a
“fast” suite of tests)

That doesn’t mean we cannot use Travis for the 90 other repos we have in
order to get test results of PRs from non-committers.


> T
>
> On Thu, Jan 3, 2019 at 8:39 PM Stephen Connolly <
> stephen.alan.conno...@gmail.com> wrote:
>
> > On Thu 3 Jan 2019 at 18:03, Enrico Olivelli  wrote:
> >
> > > Il gio 3 gen 2019, 17:38 Mickael Istria  ha
> scritto:
> > >
> > > > Hi,
> > > >
> > > > I think this discussion is diverging into "trying TravisCI for some
> > > > plugins" and is loosing focus on the initial question of how to
> improve
> > > the
> > > > build+test flow to get faster feedback as a contributor or as a
> > reviewer.
> >
> >
> > Travis will get the build+test flow faster for non-committers.
> >
> >
> > > > Can the GitHub PR builder plugin be enabled on Maven Core ?
> >
> >
> > No, it’s not compatible with how we build, as currently we only build
> from
> > gitbox not from GitHub so there is no link for Jenkins to see
> >
> >
> > Committers
> > > > would be allowed to trigger a test with a single comment once they
> > > checked
> > > > it doesn't cause a security flaw.
> > > >
> > >
> > > It turns out that it is not simple as it could seem because we have
> > custom
> > > Jenkins plugins to scan all the repos and have a common configuration,
> so
> > > in order to achieve such goal we need to enhance that plugin, please
> > > Stephen correct me if I am wrong.
> >
> >
> > Yep I need to enhance the plugin a little... just trying to clear stuff
> off
> > my plate
> >
> >
> > > Travis jumped on this train because it is easy to enable and it is very
> > > widespread, and it leaves security problems out of ASF infra.
> > > But a check with Travis won't ever cover all jobs we are actually
> > starting
> > > per-branch.
> > > It can be a good compromise to start, but if we have resources to
> improve
> > > current integration this will be the best choice for the mid term.
> > >
> > > Enrico
> > >
> > > >
> > > > Cheers,
> > > >
> > > --
> > >
> > >
> > > -- Enrico Olivelli
> > >
> > --
> > Sent from my phone
> >
>
-- 
Sent from my phone

Re: New Committers - community

[Enrico Olivelli]

Il gio 3 gen 2019, 21:52 Tibor Digana  ha scritto:

> I already lost the point of this thread.
>

We are trying to figure out how to give better support for new
contributions, the first point is to give feedback as soon as possible (an
automatic build is the top)

Enrico


Still the disk space problem on Windows executors in ASF Jenkins?
> If it's this issue, then why the workspace is not investigated directly on
> the system with remote access?
> Jenkins is very good tool and I do not want to use Travis but Travis is one
> step after finding the root cause.
> T
>
> On Thu, Jan 3, 2019 at 8:39 PM Stephen Connolly <
> stephen.alan.conno...@gmail.com> wrote:
>
> > On Thu 3 Jan 2019 at 18:03, Enrico Olivelli  wrote:
> >
> > > Il gio 3 gen 2019, 17:38 Mickael Istria  ha
> scritto:
> > >
> > > > Hi,
> > > >
> > > > I think this discussion is diverging into "trying TravisCI for some
> > > > plugins" and is loosing focus on the initial question of how to
> improve
> > > the
> > > > build+test flow to get faster feedback as a contributor or as a
> > reviewer.
> >
> >
> > Travis will get the build+test flow faster for non-committers.
> >
> >
> > > > Can the GitHub PR builder plugin be enabled on Maven Core ?
> >
> >
> > No, it’s not compatible with how we build, as currently we only build
> from
> > gitbox not from GitHub so there is no link for Jenkins to see
> >
> >
> > Committers
> > > > would be allowed to trigger a test with a single comment once they
> > > checked
> > > > it doesn't cause a security flaw.
> > > >
> > >
> > > It turns out that it is not simple as it could seem because we have
> > custom
> > > Jenkins plugins to scan all the repos and have a common configuration,
> so
> > > in order to achieve such goal we need to enhance that plugin, please
> > > Stephen correct me if I am wrong.
> >
> >
> > Yep I need to enhance the plugin a little... just trying to clear stuff
> off
> > my plate
> >
> >
> > > Travis jumped on this train because it is easy to enable and it is very
> > > widespread, and it leaves security problems out of ASF infra.
> > > But a check with Travis won't ever cover all jobs we are actually
> > starting
> > > per-branch.
> > > It can be a good compromise to start, but if we have resources to
> improve
> > > current integration this will be the best choice for the mid term.
> > >
> > > Enrico
> > >
> > > >
> > > > Cheers,
> > > >
> > > --
> > >
> > >
> > > -- Enrico Olivelli
> > >
> > --
> > Sent from my phone
> >
>
-- 


-- Enrico Olivelli

Re: New Committers - community

[Tibor Digana]

I already lost the point of this thread.
Still the disk space problem on Windows executors in ASF Jenkins?
If it's this issue, then why the workspace is not investigated directly on
the system with remote access?
Jenkins is very good tool and I do not want to use Travis but Travis is one
step after finding the root cause.
T

On Thu, Jan 3, 2019 at 8:39 PM Stephen Connolly <
stephen.alan.conno...@gmail.com> wrote:

> On Thu 3 Jan 2019 at 18:03, Enrico Olivelli  wrote:
>
> > Il gio 3 gen 2019, 17:38 Mickael Istria  ha scritto:
> >
> > > Hi,
> > >
> > > I think this discussion is diverging into "trying TravisCI for some
> > > plugins" and is loosing focus on the initial question of how to improve
> > the
> > > build+test flow to get faster feedback as a contributor or as a
> reviewer.
>
>
> Travis will get the build+test flow faster for non-committers.
>
>
> > > Can the GitHub PR builder plugin be enabled on Maven Core ?
>
>
> No, it’s not compatible with how we build, as currently we only build from
> gitbox not from GitHub so there is no link for Jenkins to see
>
>
> Committers
> > > would be allowed to trigger a test with a single comment once they
> > checked
> > > it doesn't cause a security flaw.
> > >
> >
> > It turns out that it is not simple as it could seem because we have
> custom
> > Jenkins plugins to scan all the repos and have a common configuration, so
> > in order to achieve such goal we need to enhance that plugin, please
> > Stephen correct me if I am wrong.
>
>
> Yep I need to enhance the plugin a little... just trying to clear stuff off
> my plate
>
>
> > Travis jumped on this train because it is easy to enable and it is very
> > widespread, and it leaves security problems out of ASF infra.
> > But a check with Travis won't ever cover all jobs we are actually
> starting
> > per-branch.
> > It can be a good compromise to start, but if we have resources to improve
> > current integration this will be the best choice for the mid term.
> >
> > Enrico
> >
> > >
> > > Cheers,
> > >
> > --
> >
> >
> > -- Enrico Olivelli
> >
> --
> Sent from my phone
>

Re: New Committers - community

[Stephen Connolly]

On Thu 3 Jan 2019 at 18:03, Enrico Olivelli  wrote:

> Il gio 3 gen 2019, 17:38 Mickael Istria  ha scritto:
>
> > Hi,
> >
> > I think this discussion is diverging into "trying TravisCI for some
> > plugins" and is loosing focus on the initial question of how to improve
> the
> > build+test flow to get faster feedback as a contributor or as a reviewer.


Travis will get the build+test flow faster for non-committers.


> > Can the GitHub PR builder plugin be enabled on Maven Core ?


No, it’s not compatible with how we build, as currently we only build from
gitbox not from GitHub so there is no link for Jenkins to see


Committers
> > would be allowed to trigger a test with a single comment once they
> checked
> > it doesn't cause a security flaw.
> >
>
> It turns out that it is not simple as it could seem because we have custom
> Jenkins plugins to scan all the repos and have a common configuration, so
> in order to achieve such goal we need to enhance that plugin, please
> Stephen correct me if I am wrong.


Yep I need to enhance the plugin a little... just trying to clear stuff off
my plate


> Travis jumped on this train because it is easy to enable and it is very
> widespread, and it leaves security problems out of ASF infra.
> But a check with Travis won't ever cover all jobs we are actually starting
> per-branch.
> It can be a good compromise to start, but if we have resources to improve
> current integration this will be the best choice for the mid term.
>
> Enrico
>
> >
> > Cheers,
> >
> --
>
>
> -- Enrico Olivelli
>
-- 
Sent from my phone

Re: New Committers - community

[Enrico Olivelli]

Il gio 3 gen 2019, 17:38 Mickael Istria  ha scritto:

> Hi,
>
> I think this discussion is diverging into "trying TravisCI for some
> plugins" and is loosing focus on the initial question of how to improve the
> build+test flow to get faster feedback as a contributor or as a reviewer.
> Can the GitHub PR builder plugin be enabled on Maven Core ? Committers
> would be allowed to trigger a test with a single comment once they checked
> it doesn't cause a security flaw.
>

It turns out that it is not simple as it could seem because we have custom
Jenkins plugins to scan all the repos and have a common configuration, so
in order to achieve such goal we need to enhance that plugin, please
Stephen correct me if I am wrong.
Travis jumped on this train because it is easy to enable and it is very
widespread, and it leaves security problems out of ASF infra.
But a check with Travis won't ever cover all jobs we are actually starting
per-branch.
It can be a good compromise to start, but if we have resources to improve
current integration this will be the best choice for the mid term.

Enrico

>
> Cheers,
>
-- 


-- Enrico Olivelli

Re: New Committers - community

[Mickael Istria]

Hi,

I think this discussion is diverging into "trying TravisCI for some
plugins" and is loosing focus on the initial question of how to improve the
build+test flow to get faster feedback as a contributor or as a reviewer.
Can the GitHub PR builder plugin be enabled on Maven Core ? Committers
would be allowed to trigger a test with a single comment once they checked
it doesn't cause a security flaw.

Cheers,

Re: New Committers - community

[Enrico Olivelli]

Il gio 3 gen 2019, 08:19 Hervé BOUTEMY  ha scritto:

> yes, on our 100 Git repos [1], 90% should be easy to run on any CI with
> minimum configuration and resources
>
> then there is Maven core and complex plugins (like Surefire) which are not
> the
> same story
>
> if someone wants to test other toolings to show some useful feature we
> don't
> get in our current ASF Jenkins configuration [2], just start with some
> examples of the base 90% and make a demo of something that would be useful
> to
> add to our configuration
>

I would like to try
Maybe a brand new plugin like [3] scripting plugin can be a good candidate
But we need a configuration from infra and I don't know if we need a formal
vote or at least some PMC formal approval

Enrico


[3] https://github.com/apache/maven-scripting-plugin/pull/1


> Regards,
>
> Hervé
>
> [1] https://maven.apache.org/scm.html#Maven_Sources_Overview
>
> [2] https://github.com/apache/maven-jenkins-lib
>
> Le mercredi 2 janvier 2019, 15:38:12 CET Enrico Olivelli a écrit :
> > Il giorno mer 2 gen 2019 alle ore 15:34 Vladimir Sitnikov
> >
> >  ha scritto:
> > > Enrico> For instance maven-surefire
> > > integration tests take 2 hours.
> > > In general I see Travis (free edition) does not offer many resources.
> > >
> > > Each Travis job is limited by 50 minutes, however one can split tests
> into
> > > multiple jobs, so it will run fine.
> >
> > I guess it won't be easy.
> > Maven integration testing is very complex, it's Maven launching Maven
> > itself. We will need to improve maven-invoker plugin, I don't know if
> there
> > is support for something like JUnit categories.
> > We can run only "unit tests" and this will be a minimal "feedback" for
> > the contributor.
> > For simpler plugins, like "checkstyle", I think Travis would be a good
> > choice, at least I would give it a try
> >
> > Enrico
> >
> > > Vladimir
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > For additional commands, e-mail: dev-h...@maven.apache.org
>
>
>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> For additional commands, e-mail: dev-h...@maven.apache.org
>
> --


-- Enrico Olivelli

Re: New Committers - community

[Hervé BOUTEMY]

yes, on our 100 Git repos [1], 90% should be easy to run on any CI with 
minimum configuration and resources

then there is Maven core and complex plugins (like Surefire) which are not the 
same story

if someone wants to test other toolings to show some useful feature we don't 
get in our current ASF Jenkins configuration [2], just start with some 
examples of the base 90% and make a demo of something that would be useful to 
add to our configuration

Regards,

Hervé

[1] https://maven.apache.org/scm.html#Maven_Sources_Overview

[2] https://github.com/apache/maven-jenkins-lib

Le mercredi 2 janvier 2019, 15:38:12 CET Enrico Olivelli a écrit :
> Il giorno mer 2 gen 2019 alle ore 15:34 Vladimir Sitnikov
> 
>  ha scritto:
> > Enrico> For instance maven-surefire
> > integration tests take 2 hours.
> > In general I see Travis (free edition) does not offer many resources.
> > 
> > Each Travis job is limited by 50 minutes, however one can split tests into
> > multiple jobs, so it will run fine.
> 
> I guess it won't be easy.
> Maven integration testing is very complex, it's Maven launching Maven
> itself. We will need to improve maven-invoker plugin, I don't know if there
> is support for something like JUnit categories.
> We can run only "unit tests" and this will be a minimal "feedback" for
> the contributor.
> For simpler plugins, like "checkstyle", I think Travis would be a good
> choice, at least I would give it a try
> 
> Enrico
> 
> > Vladimir
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> For additional commands, e-mail: dev-h...@maven.apache.org





-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: New Committers - community

[Enrico Olivelli]

Il giorno mer 2 gen 2019 alle ore 15:34 Vladimir Sitnikov
 ha scritto:
>
> Enrico> For instance maven-surefire
> integration tests take 2 hours.
> In general I see Travis (free edition) does not offer many resources.
>
> Each Travis job is limited by 50 minutes, however one can split tests into
> multiple jobs, so it will run fine.

I guess it won't be easy.
Maven integration testing is very complex, it's Maven launching Maven itself.
We will need to improve maven-invoker plugin, I don't know if there is
support for something like JUnit categories.
We can run only "unit tests" and this will be a minimal "feedback" for
the contributor.
For simpler plugins, like "checkstyle", I think Travis would be a good
choice, at least I would give it a try

Enrico

>
> Vladimir

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: New Committers - community

[Vladimir Sitnikov]

Enrico> For instance maven-surefire
integration tests take 2 hours.
In general I see Travis (free edition) does not offer many resources.

Each Travis job is limited by 50 minutes, however one can split tests into
multiple jobs, so it will run fine.

Vladimir

Re: New Committers - community

[Enrico Olivelli]

Il giorno mer 2 gen 2019 alle ore 15:12 Vladimir Sitnikov
 ha scritto:
>
> Enrico> Probably Travis won't be able to run our ITs, at least the free
> version
>
> Why do you think so?

Because Maven CI jobs are huge. For instance maven-surefire
integration tests take 2 hours.
In general I see Travis (free edition) does not offer many resources.

I would like to try to setup Travis for some "small" plugin and make
some test, at least testing java 11 + Linux

Enrico

>
> Vladimir

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: New Committers - community

[Vladimir Sitnikov]

Enrico> Probably Travis won't be able to run our ITs, at least the free
version

Why do you think so?

Vladimir

Re: New Committers - community

[Enrico Olivelli]

Il dom 30 dic 2018, 00:53 Mickael Istria  ha scritto:

> On Saturday, December 29, 2018, Stephen Connolly <
> stephen.alan.conno...@gmail.com> wrote:
> >
> > I think we could easily use TravisCI or one of the cloud CI vendors to
> > perform trial builds of PRs on GitHub.
>
>
> Good. So at this point, it's only a matter of writing a .travis.yaml file
> in the repo, or is there anything else missing?
>

Probably Travis won't be able to run our ITs, at least the free version

Enrico

>
>
> > What is of importance to the ASF is
> > that the build of ASF hosted code are on ASF hosted hardware... until the
> > PR is merged it is not ASF hosted code, it exists only on GitHub.
> >
>
> From contributor and user POV, I think where the software is built and
> where the code is hosted isn't important. So no issue if it's on ASF
> Jenkins and hardware, but for PR trials, it seems like a constraint it'd be
> sane to get rid of.
>
>
> --
> Mickael Istria
> Eclipse IDE 
> developer, for Red Hat Developers 
>
-- 


-- Enrico Olivelli

Re: New Committers - community

[Mickael Istria]

On Saturday, December 29, 2018, Stephen Connolly <
stephen.alan.conno...@gmail.com> wrote:
>
> I think we could easily use TravisCI or one of the cloud CI vendors to
> perform trial builds of PRs on GitHub.


Good. So at this point, it's only a matter of writing a .travis.yaml file
in the repo, or is there anything else missing?


> What is of importance to the ASF is
> that the build of ASF hosted code are on ASF hosted hardware... until the
> PR is merged it is not ASF hosted code, it exists only on GitHub.
>

>From contributor and user POV, I think where the software is built and
where the code is hosted isn't important. So no issue if it's on ASF
Jenkins and hardware, but for PR trials, it seems like a constraint it'd be
sane to get rid of.


-- 
Mickael Istria
Eclipse IDE 
developer, for Red Hat Developers 

Re: New Committers - community

[Vladimir Sitnikov]

Stephen > Are you sure?

I've just created a dummy account and "approval" does not make merge button
magically appear.

If you have a couple of minutes, please make a PR (I'll approve it shortly)
to https://github.com/vlsi/KE-complex_modifications
Note: you can click "edit" button in GitHub UI for a random file, so you
don't really need to clone the project.

I've attached no_merge.png which is a screenshot of GitHub UI for PR
author. Apparently, there's no merge button.

Vladimir

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: New Committers - community

[Vladimir Sitnikov]

Karl Heinz Marbaise > The user community is very big but unfortunately the
people who are
Karl Heinz Marbaise > willing to help is not that big...

Can I (ab)use the thread a bit?

One of the issues I run into with Maven is "it takes ages for repeated
build when nothing has changed".
Of course there are details, yet the final perception is like that.

I know Apache JMeter team sticks with Ant for a mere reason of fast builds
(even though it is way harder to configure in IDE)

Once upon a time I was bitten hard enough by Maven which always rebuilds
everything even for repeated builds, so I investigated the reason and
created https://issues.apache.org/jira/browse/CALCITE-484
The basic case is "jar is rebuilt due to resource was somehow modified".
Of course there are multiple offenders, however two cases come from
maven-resources-plugin and maven-remote-resources-plugin.

I've created a PR (in December 2014) to fix maven-remote-resources-plugin:
https://github.com/apache/maven-plugins/pull/40 (Avoid overwrite of the
destination file if the produced contents is the same), and it took just
FOUR years to release the new version of the plugin in October 2018.

The second offender is maven-filtering (
https://issues.apache.org/jira/browse/MSHARED-394 )
In a brief, maven-filtering rewrites the file not matter the contents. For
instance, when Maven copies resources, it compares timestamps, so it can
skip copy if the file is good enough.
However maven-filtering always writes the destination file, thus it
triggers jar rebuild, etc.

Frankly speaking, I'm proud of a trick in
https://github.com/apache/maven-plugins/pull/40 which keeps contents in
memory, and if the buffer overflows, then it writes the contents to the
destination file. In other words, no temporary files are required, and it
can skip file overwrites when the produced contents is the same.

I think similar should be done to DefaultMavenFileFilter, however Kristian
suggested he wanted to look on his own:

https://issues.apache.org/jira/browse/MSHARED-394?focusedCommentId=14449864=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14449864

>Kristian Rosenvold added a comment - 16/Dec/2014 12:19
>Haven given this issue some thought I believe I have come up with a
radically better solution than what is suggested here.
>There is no point in working on the changes suggested here,
>at least not until I give up my other plan.
>Since I am currently working on a slightly different task, it will be a
couple of weeks before I'll make any activity on this issue.


Frankly speaking, I'm very puzzled.
The net result is: my first Maven contribution (MRRESOURCES-91) took 4
years to release, and very similar one was rejected with "There is no point
in working on the changes suggested here" comment.
That is quite sad to be honest.

Vladimir

Re: New Committers - community

[Stephen Connolly]

On Sat, 29 Dec 2018 at 20:41, Mickael Istria  wrote:

> I'mglad to see that PR build/merge discussed, it seems to have a good
> potential value for many in simplifying it.
>
> FWIW, at Eclipse Foundation, similar questions were faced and risks
> identified. The result is that at the moment, all PRs from anyone can be
> built on Eclipse infra without much visible restriction. The Foundation
> might have extra guards under the hood but it's not something we developers
> need to care about.
> While this is probably unsafe, it's so far so good. We weren't warned by
> the Foundation about malicious usage of this permissive access to build
> machines.
>

As one of the primary developers of Jenkins and as a member of the Jenkins
CERT team I cannot and do not endorse that viewpoint. Unless and until ASF
infra has throw-away build machines, I do not recommend running CI builds
of PRs from unknown actors (it being trivial to set up a throw-away GitHub
account) on the ASF dedicated build agents. Eclipse is being fools if they
are allowing similar.


> AFAIK, there is no issue from developer POV about GitHub API rates limit.
> I suggest the Apache infra team gets in touch with Eclipse Foundation one
> to sort out whether similar configurations could be implemented in a
> safe-enough way.
>
> The GitHub PR builder plugin already has support for whitelisting users and
> giving them ability to trigger a build from a non-whitelidsted contributor
> with a single «test PR» comment or similar.
> Then build progress and report are reported as expected.
> And, again, TravisCI also does the same in a decent way, with the benefit
> of worrying less about underlying infra and security, and only drawback of
> being discoupled from a specific infra (is it really a drawback?)
>

I think we could easily use TravisCI or one of the cloud CI vendors to
perform trial builds of PRs on GitHub. What is of importance to the ASF is
that the build of ASF hosted code are on ASF hosted hardware... until the
PR is merged it is not ASF hosted code, it exists only on GitHub.


>
> Cheers,
>
>
> --
> Mickael Istria
> Eclipse IDE 
> developer, for Red Hat Developers 
>

Re: New Committers - community

[Stephen Connolly]

On Sat, 29 Dec 2018 at 18:56, Vladimir Sitnikov 
wrote:

> Stephen> Well on other GitHub orgs i’ve seen PR author has Merge
> button once PR is
> Stephen> approved by someone with push rights to the repo... until
> they add a commit
> Stephen> or the merge result changes
>
> It does not work the way you describe.
>
> 1) By default GitHub defaults to "Base permissions" == "read".
> Administrator can set even "base permissions == admin", however that
> is not something that comes by default.
> 2) Repository administrator can configure a branch so it requires a PR
> approval before merge (see
>
> https://help.github.com/articles/enabling-required-reviews-for-pull-requests/
> )
>
> It might happen so that "author has Merge button" is a case when
> repository was configured with "default=write" permissions + "require
> PR approval".
> Indeed that should produce (I have not checked) the behavior you
> describe, however it was never a default mode for GiHub.
>
>
Are you sure? I have lots of repos on different orgs that I am in and for
almost all of them, if the repo is one where I only have read access I can
create the PR but the merge button is unavailable... until the PR is
approved by somebody *who has write permission*... at which point the merge
button magically becomes available to me...

Now you have to know to look out for it, because typically the owner of the
repo approves and then merges the PR in one go... and if I add a new commit
or resolve a merge conflict I lose the merge button until re-approved.

But for some organizations I am involved with, we have the culture of
allowing you to self-merge.

Oh and I know for a fact that these are at least some repos that do not
have branch protection turned on.


> Vladimir
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> For additional commands, e-mail: dev-h...@maven.apache.org
>
>

Re: New Committers - community

[Vladimir Sitnikov]

> And, again, TravisCI also does the same in a decent way, with the benefit
> of worrying less about underlying infra and security, and only drawback of
> being discoupled from a specific infra (is it really a drawback?)

Just in case: Apache Calcite uses both Travis CI and AppVeryor. It just
works.

It does simplify validation of the changes across various Java versions,
javadoc, RAT.
AppVeyor validates Windows builds.

Both results are automatically displayed at GitHub PR (e.g.
https://github.com/apache/calcite/pull/976) with no extra manual actions.

Vladimir

Re: New Committers - community

[Mickael Istria]

I'mglad to see that PR build/merge discussed, it seems to have a good
potential value for many in simplifying it.

FWIW, at Eclipse Foundation, similar questions were faced and risks
identified. The result is that at the moment, all PRs from anyone can be
built on Eclipse infra without much visible restriction. The Foundation
might have extra guards under the hood but it's not something we developers
need to care about.
While this is probably unsafe, it's so far so good. We weren't warned by
the Foundation about malicious usage of this permissive access to build
machines.
AFAIK, there is no issue from developer POV about GitHub API rates limit.
I suggest the Apache infra team gets in touch with Eclipse Foundation one
to sort out whether similar configurations could be implemented in a
safe-enough way.

The GitHub PR builder plugin already has support for whitelisting users and
giving them ability to trigger a build from a non-whitelidsted contributor
with a single «test PR» comment or similar.
Then build progress and report are reported as expected.
And, again, TravisCI also does the same in a decent way, with the benefit
of worrying less about underlying infra and security, and only drawback of
being discoupled from a specific infra (is it really a drawback?)

Cheers,


-- 
Mickael Istria
Eclipse IDE 
developer, for Red Hat Developers 

Re: New Committers - community

[Vladimir Sitnikov]

Stephen> Well on other GitHub orgs i’ve seen PR author has Merge
button once PR is
Stephen> approved by someone with push rights to the repo... until
they add a commit
Stephen> or the merge result changes

It does not work the way you describe.

1) By default GitHub defaults to "Base permissions" == "read".
Administrator can set even "base permissions == admin", however that
is not something that comes by default.
2) Repository administrator can configure a branch so it requires a PR
approval before merge (see
https://help.github.com/articles/enabling-required-reviews-for-pull-requests/
)

It might happen so that "author has Merge button" is a case when
repository was configured with "default=write" permissions + "require
PR approval".
Indeed that should produce (I have not checked) the behavior you
describe, however it was never a default mode for GiHub.

Vladimir

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: New Committers - community

[Stephen Connolly]

On Sat 29 Dec 2018 at 18:19, Stephen Connolly <
stephen.alan.conno...@gmail.com> wrote:

>
>
> On Sat 29 Dec 2018 at 18:06, Vladimir Sitnikov <
> sitnikov.vladi...@gmail.com> wrote:
>
>> Stephen> Nah. Once committers have approved the PR then the PR can be
>> merged by the
>> Stephen> creator of the PR even if not a committer... at least that’s
>> the default
>> Stephen> way GitHub PRs work
>>
>> By default one needs push rights on the repository to merge PRs.
>> "Approval" changes nothing. "PR approval" does not differ from any
>> other comment.
>> In other words, non-committers can't merge PRs.
>
>
> Well on other GitHub orgs i’ve seen PR author has Merge button once PR is
> approved by someone with push rights to the repo... until they add a commit
> or the merge result changes
>

Gitbox may be configuring synchronised repos differently, but for interop
with the rest of GitHub we shouldn’t assume approval of a PR by committers
will not enable self-merging


>>
>> Vladimir
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
>> For additional commands, e-mail: dev-h...@maven.apache.org
>>
>> --
> Sent from my phone
>
-- 
Sent from my phone

Re: New Committers - community

[Stephen Connolly]

On Sat 29 Dec 2018 at 18:06, Vladimir Sitnikov 
wrote:

> Stephen> Nah. Once committers have approved the PR then the PR can be
> merged by the
> Stephen> creator of the PR even if not a committer... at least that’s
> the default
> Stephen> way GitHub PRs work
>
> By default one needs push rights on the repository to merge PRs.
> "Approval" changes nothing. "PR approval" does not differ from any
> other comment.
> In other words, non-committers can't merge PRs.


Well on other GitHub orgs i’ve seen PR author has Merge button once PR is
approved by someone with push rights to the repo... until they add a commit
or the merge result changes

>
>
> Vladimir
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> For additional commands, e-mail: dev-h...@maven.apache.org
>
> --
Sent from my phone

Re: New Committers - community

[Vladimir Sitnikov]

Stephen> Nah. Once committers have approved the PR then the PR can be
merged by the
Stephen> creator of the PR even if not a committer... at least that’s
the default
Stephen> way GitHub PRs work

By default one needs push rights on the repository to merge PRs.
"Approval" changes nothing. "PR approval" does not differ from any
other comment.
In other words, non-committers can't merge PRs.

Vladimir

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: New Committers - community

[Stephen Connolly]

On Sat 29 Dec 2018 at 17:16, Enrico Olivelli  wrote:

> Il sab 29 dic 2018, 17:25 Stephen Connolly <
> stephen.alan.conno...@gmail.com>
> ha scritto:
>
> > On Sat 29 Dec 2018 at 16:20, Stephen Connolly <
> > stephen.alan.conno...@gmail.com> wrote:
> >
> > >
> > >
> > > On Sat 29 Dec 2018 at 15:18, Enrico Olivelli 
> > wrote:
> > >
> > >> Il sab 29 dic 2018, 15:17 Stephen Connolly <
> > >> stephen.alan.conno...@gmail.com>
> > >> ha scritto:
> > >>
> > >> > There is a security issue with building PRs automatically.
> > >> >
> > >> > I can see about adding PR discovery to the existing ASF gitbox
> plugin,
> > >> but
> > >> > we’d need committers to ok the build and have reviewed the code as
> the
> > >> PR
> > >> > could contain attacks to be run from ASF hardware... which is why we
> > >> don’t
> > >> > build PRs at present.
> > >> >
> > >>
> > >> Now I have to review and then push to ASF repo and I have to tell to
> the
> > >> contributor that I will make CI start.
> > >> IMHO a good tradeoff is:
> > >> - a committer adds a 'test this please' comment, or '@asfbot test this
> > >> please' and then a CI job start
> > >> - this job updates the status line of the PR, with a link to the logs
> > and
> > >> the status of the job
> > >>
> > >> How does it sounds to you?
> > >
> > >
> > > Like it’ll burn through the GitHub api rate limit like crazy.
> >
>
> I did not think we have 100 repos
>
> > >
> > > The committer goes to Jenkins and clicks the build button on the PR job
> > > (which is sitting there ready and waiting), done.
> > >
> >
> > Oh and before I forget, clicking build now I’m Jenkins will update the CI
> > status in GitHub for the PR to say in progress and then provide the
> result
> > when available.
> >
> >
> > > Searching through comments on PRs to find commits with a magic comment
> > > string that haven’t been built by Jenkins is certainly what would be
> > > lovely... but right now it would burn the GitHub rate limit (which is
> > 5,000
> > > requests per hour across the whole ASF) right through.
> > >
> >
> > To be clear, the hard part is efficiently finding “missed” comments and
> > associating with the commit hash they belong to. We don’t want an
> approval
> > to allow attack via an update pushed to the PR. So you have to walk all
> the
> > comments and associate with the commit hash they applied to... gets
> tricky.
> >
>
> Yep, I understand, you are right.
>
> Another option is to have a script to launch:
>
> import-pr maven-assembly-plugin #567
>
> Then the script + Jenkins:
> - bind the pr to a JIRA by scanning git log
> - push to ASF (changing 'committer') with a standard name (JIRA id)
> - start a job
> - add a github status line with a link to the logs
> - bonus: the job will change the status line with green/red light
>
> I already have such kind of script in my company (but for
> bitbucket/JIRA/Jenkins and not for such a complex system like Maven CI),
> but the hard part is the job which has to write the status line
>
> Enrico
>
>
> > We could maybe hijack approval state... but that allows merging by the
> > author.
> >
>
> This part is not clear to me. Only 'commiters' can push to the repo.


Nah. Once committers have approved the PR then the PR can be merged by the
creator of the PR even if not a committer... at least that’s the default
way GitHub PRs work. GitBox *may* be enforcing a stricter permissions
model... but I suspect not (because it makes contributions harder... and
the goal of gitbox was to make working from GitHub a first class experience)




>
>
>
> >
> > >
> > >
> > >>
> > >> Enrico
> > >>
> > >>
> > >> > Other projects at ASF probably missed this point in the video series
> > >> > chronicling the development of the plugin...
> > >> >
> > >> > On Sat 29 Dec 2018 at 13:10, Enrico Olivelli 
> > >> wrote:
> > >> >
> > >> > > Hervè,
> > >> > > This is the plugin
> > >> > >
> > >> > >
> > >> >
> > >>
> >
> https://wiki.jenkins.io/display/JENKINS/GitHub+Branch+Source+Plugin?desktop=true=unmigrated-inline-wiki-markup
> > >> > >
> > >> > > I see our "maven-box" is using some special "Scan Apache Hosted
> Git
> > >> > > Folder Triggers" source
> > >> > > (https://builds.apache.org/job/maven-box/configure)
> > >> > > In the Jenkins of my company in a "Multibranch Pipeline" I have a
> > >> > > "Branch Sources" box with a "GitHub" option which lets me trigger
> > >> > > builds by using PRs
> > >> > >
> > >> > >
> > >> > > Enrico
> > >> > >
> > >> > > Il giorno sab 29 dic 2018 alle ore 13:43 Hervé BOUTEMY
> > >> > >  ha scritto:
> > >> > > >
> > >> > > > Le samedi 29 décembre 2018, 12:40:20 CET Enrico Olivelli a
> écrit :
> > >> > > > > Il sab 29 dic 2018, 12:37 Mickael Istria 
> > ha
> > >> > > scritto:
> > >> > > > > > On Sat, Dec 29, 2018 at 12:01 PM Hervé BOUTEMY <
> > >> > > herve.bout...@free.fr>
> > >> > > > > >
> > >> > > > > > wrote:
> > >> > > > > > > But in both cases, currently, there is no automatic GitHub
> > PR
> > >> > > > > >
> > >> > > > > > integration:
> > 

Re: New Committers - community

[Karl Heinz Marbaise]

Hi,

On 29/12/18 18:16, Enrico Olivelli wrote:

Il sab 29 dic 2018, 17:25 Stephen Connolly 
ha scritto:


On Sat 29 Dec 2018 at 16:20, Stephen Connolly <
stephen.alan.conno...@gmail.com> wrote:




On Sat 29 Dec 2018 at 15:18, Enrico Olivelli 

wrote:



Il sab 29 dic 2018, 15:17 Stephen Connolly <
stephen.alan.conno...@gmail.com>
ha scritto:


There is a security issue with building PRs automatically.

I can see about adding PR discovery to the existing ASF gitbox plugin,

but

we’d need committers to ok the build and have reviewed the code as the

PR

could contain attacks to be run from ASF hardware... which is why we

don’t

build PRs at present.



Now I have to review and then push to ASF repo and I have to tell to the
contributor that I will make CI start.
IMHO a good tradeoff is:
- a committer adds a 'test this please' comment, or '@asfbot test this
please' and then a CI job start
- this job updates the status line of the PR, with a link to the logs

and

the status of the job

How does it sounds to you?



Like it’ll burn through the GitHub api rate limit like crazy.




I did not think we have 100 repos


I wouldn't be that sure:

https://builds.apache.org/view/M-R/view/Maven/job/maven-box/
(96 at the moment)..

or https://gitbox.apache.org/repos/asf

search for "Apache Maven"...

Apart from that the GitHub API rate limit is for the whole ASF 
orga...not only for our project...


Kind regards
Karl Heinz Marbaise

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: New Committers - community

[Enrico Olivelli]

Il sab 29 dic 2018, 17:25 Stephen Connolly 
ha scritto:

> On Sat 29 Dec 2018 at 16:20, Stephen Connolly <
> stephen.alan.conno...@gmail.com> wrote:
>
> >
> >
> > On Sat 29 Dec 2018 at 15:18, Enrico Olivelli 
> wrote:
> >
> >> Il sab 29 dic 2018, 15:17 Stephen Connolly <
> >> stephen.alan.conno...@gmail.com>
> >> ha scritto:
> >>
> >> > There is a security issue with building PRs automatically.
> >> >
> >> > I can see about adding PR discovery to the existing ASF gitbox plugin,
> >> but
> >> > we’d need committers to ok the build and have reviewed the code as the
> >> PR
> >> > could contain attacks to be run from ASF hardware... which is why we
> >> don’t
> >> > build PRs at present.
> >> >
> >>
> >> Now I have to review and then push to ASF repo and I have to tell to the
> >> contributor that I will make CI start.
> >> IMHO a good tradeoff is:
> >> - a committer adds a 'test this please' comment, or '@asfbot test this
> >> please' and then a CI job start
> >> - this job updates the status line of the PR, with a link to the logs
> and
> >> the status of the job
> >>
> >> How does it sounds to you?
> >
> >
> > Like it’ll burn through the GitHub api rate limit like crazy.
>

I did not think we have 100 repos

> >
> > The committer goes to Jenkins and clicks the build button on the PR job
> > (which is sitting there ready and waiting), done.
> >
>
> Oh and before I forget, clicking build now I’m Jenkins will update the CI
> status in GitHub for the PR to say in progress and then provide the result
> when available.
>
>
> > Searching through comments on PRs to find commits with a magic comment
> > string that haven’t been built by Jenkins is certainly what would be
> > lovely... but right now it would burn the GitHub rate limit (which is
> 5,000
> > requests per hour across the whole ASF) right through.
> >
>
> To be clear, the hard part is efficiently finding “missed” comments and
> associating with the commit hash they belong to. We don’t want an approval
> to allow attack via an update pushed to the PR. So you have to walk all the
> comments and associate with the commit hash they applied to... gets tricky.
>

Yep, I understand, you are right.

Another option is to have a script to launch:

import-pr maven-assembly-plugin #567

Then the script + Jenkins:
- bind the pr to a JIRA by scanning git log
- push to ASF (changing 'committer') with a standard name (JIRA id)
- start a job
- add a github status line with a link to the logs
- bonus: the job will change the status line with green/red light

I already have such kind of script in my company (but for
bitbucket/JIRA/Jenkins and not for such a complex system like Maven CI),
but the hard part is the job which has to write the status line

Enrico


> We could maybe hijack approval state... but that allows merging by the
> author.
>

This part is not clear to me. Only 'commiters' can push to the repo.



>
> >
> >
> >>
> >> Enrico
> >>
> >>
> >> > Other projects at ASF probably missed this point in the video series
> >> > chronicling the development of the plugin...
> >> >
> >> > On Sat 29 Dec 2018 at 13:10, Enrico Olivelli 
> >> wrote:
> >> >
> >> > > Hervè,
> >> > > This is the plugin
> >> > >
> >> > >
> >> >
> >>
> https://wiki.jenkins.io/display/JENKINS/GitHub+Branch+Source+Plugin?desktop=true=unmigrated-inline-wiki-markup
> >> > >
> >> > > I see our "maven-box" is using some special "Scan Apache Hosted Git
> >> > > Folder Triggers" source
> >> > > (https://builds.apache.org/job/maven-box/configure)
> >> > > In the Jenkins of my company in a "Multibranch Pipeline" I have a
> >> > > "Branch Sources" box with a "GitHub" option which lets me trigger
> >> > > builds by using PRs
> >> > >
> >> > >
> >> > > Enrico
> >> > >
> >> > > Il giorno sab 29 dic 2018 alle ore 13:43 Hervé BOUTEMY
> >> > >  ha scritto:
> >> > > >
> >> > > > Le samedi 29 décembre 2018, 12:40:20 CET Enrico Olivelli a écrit :
> >> > > > > Il sab 29 dic 2018, 12:37 Mickael Istria 
> ha
> >> > > scritto:
> >> > > > > > On Sat, Dec 29, 2018 at 12:01 PM Hervé BOUTEMY <
> >> > > herve.bout...@free.fr>
> >> > > > > >
> >> > > > > > wrote:
> >> > > > > > > But in both cases, currently, there is no automatic GitHub
> PR
> >> > > > > >
> >> > > > > > integration:
> >> > > > > > > Maven committers need to create a branch in the official
> >> > > repository to
> >> > > > > > > benefit
> >> > > > > > > from ASF Jenkins build
> >> > > > > >
> >> > > > > > Ah ok, I wasn't aware the GitHub repo was "unofficial" and
> >> couldn't
> >> > > be
> >> > > > > > used
> >> > > > > > to trigger builds. That sucks...
> >> > > > >
> >> > > > > Maven migrated to gitbox so actually github is an official repo
> >> for
> >> > > Maven.
> >> > > > > I see the same setup in Zookeeper and Bookkeeper and github pr
> >> plugin
> >> > > works
> >> > > > > like a charm (and I partecipated in setting it up)
> >> > > > oh great, that would be nice to have the same setup for Maven
> repos!
> >> > > >
> >> > > > >
> >> > > > > Enrico

Re: New Committers - community

[Stephen Connolly]

On Sat 29 Dec 2018 at 16:20, Stephen Connolly <
stephen.alan.conno...@gmail.com> wrote:

>
>
> On Sat 29 Dec 2018 at 15:18, Enrico Olivelli  wrote:
>
>> Il sab 29 dic 2018, 15:17 Stephen Connolly <
>> stephen.alan.conno...@gmail.com>
>> ha scritto:
>>
>> > There is a security issue with building PRs automatically.
>> >
>> > I can see about adding PR discovery to the existing ASF gitbox plugin,
>> but
>> > we’d need committers to ok the build and have reviewed the code as the
>> PR
>> > could contain attacks to be run from ASF hardware... which is why we
>> don’t
>> > build PRs at present.
>> >
>>
>> Now I have to review and then push to ASF repo and I have to tell to the
>> contributor that I will make CI start.
>> IMHO a good tradeoff is:
>> - a committer adds a 'test this please' comment, or '@asfbot test this
>> please' and then a CI job start
>> - this job updates the status line of the PR, with a link to the logs and
>> the status of the job
>>
>> How does it sounds to you?
>
>
> Like it’ll burn through the GitHub api rate limit like crazy.
>
> The committer goes to Jenkins and clicks the build button on the PR job
> (which is sitting there ready and waiting), done.
>

Oh and before I forget, clicking build now I’m Jenkins will update the CI
status in GitHub for the PR to say in progress and then provide the result
when available.


> Searching through comments on PRs to find commits with a magic comment
> string that haven’t been built by Jenkins is certainly what would be
> lovely... but right now it would burn the GitHub rate limit (which is 5,000
> requests per hour across the whole ASF) right through.
>

To be clear, the hard part is efficiently finding “missed” comments and
associating with the commit hash they belong to. We don’t want an approval
to allow attack via an update pushed to the PR. So you have to walk all the
comments and associate with the commit hash they applied to... gets tricky.

We could maybe hijack approval state... but that allows merging by the
author.


>
>
>>
>> Enrico
>>
>>
>> > Other projects at ASF probably missed this point in the video series
>> > chronicling the development of the plugin...
>> >
>> > On Sat 29 Dec 2018 at 13:10, Enrico Olivelli 
>> wrote:
>> >
>> > > Hervè,
>> > > This is the plugin
>> > >
>> > >
>> >
>> https://wiki.jenkins.io/display/JENKINS/GitHub+Branch+Source+Plugin?desktop=true=unmigrated-inline-wiki-markup
>> > >
>> > > I see our "maven-box" is using some special "Scan Apache Hosted Git
>> > > Folder Triggers" source
>> > > (https://builds.apache.org/job/maven-box/configure)
>> > > In the Jenkins of my company in a "Multibranch Pipeline" I have a
>> > > "Branch Sources" box with a "GitHub" option which lets me trigger
>> > > builds by using PRs
>> > >
>> > >
>> > > Enrico
>> > >
>> > > Il giorno sab 29 dic 2018 alle ore 13:43 Hervé BOUTEMY
>> > >  ha scritto:
>> > > >
>> > > > Le samedi 29 décembre 2018, 12:40:20 CET Enrico Olivelli a écrit :
>> > > > > Il sab 29 dic 2018, 12:37 Mickael Istria  ha
>> > > scritto:
>> > > > > > On Sat, Dec 29, 2018 at 12:01 PM Hervé BOUTEMY <
>> > > herve.bout...@free.fr>
>> > > > > >
>> > > > > > wrote:
>> > > > > > > But in both cases, currently, there is no automatic GitHub PR
>> > > > > >
>> > > > > > integration:
>> > > > > > > Maven committers need to create a branch in the official
>> > > repository to
>> > > > > > > benefit
>> > > > > > > from ASF Jenkins build
>> > > > > >
>> > > > > > Ah ok, I wasn't aware the GitHub repo was "unofficial" and
>> couldn't
>> > > be
>> > > > > > used
>> > > > > > to trigger builds. That sucks...
>> > > > >
>> > > > > Maven migrated to gitbox so actually github is an official repo
>> for
>> > > Maven.
>> > > > > I see the same setup in Zookeeper and Bookkeeper and github pr
>> plugin
>> > > works
>> > > > > like a charm (and I partecipated in setting it up)
>> > > > oh great, that would be nice to have the same setup for Maven repos!
>> > > >
>> > > > >
>> > > > > Enrico
>> > > > >
>> > > > > > Any idea how we could have GitHub PR reviews without this branch
>> > > creation
>> > > > > >
>> > > > > > > by
>> > > > > > > committers, be it at ASF or somewhere else?
>> > > > > >
>> > > > > > Using TravisCI could be a solution.
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> -
>> > > > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
>> > > > For additional commands, e-mail: dev-h...@maven.apache.org
>> > > >
>> > >
>> > > -
>> > > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
>> > > For additional commands, e-mail: dev-h...@maven.apache.org
>> > >
>> > > --
>> > Sent from my phone
>> >
>> --
>>
>>
>> -- Enrico Olivelli
>>
> --
> Sent from my phone
>
-- 
Sent from my phone

Re: New Committers - community

[Stephen Connolly]

On Sat 29 Dec 2018 at 15:18, Enrico Olivelli  wrote:

> Il sab 29 dic 2018, 15:17 Stephen Connolly <
> stephen.alan.conno...@gmail.com>
> ha scritto:
>
> > There is a security issue with building PRs automatically.
> >
> > I can see about adding PR discovery to the existing ASF gitbox plugin,
> but
> > we’d need committers to ok the build and have reviewed the code as the PR
> > could contain attacks to be run from ASF hardware... which is why we
> don’t
> > build PRs at present.
> >
>
> Now I have to review and then push to ASF repo and I have to tell to the
> contributor that I will make CI start.
> IMHO a good tradeoff is:
> - a committer adds a 'test this please' comment, or '@asfbot test this
> please' and then a CI job start
> - this job updates the status line of the PR, with a link to the logs and
> the status of the job
>
> How does it sounds to you?


Like it’ll burn through the GitHub api rate limit like crazy.

The committer goes to Jenkins and clicks the build button on the PR job
(which is sitting there ready and waiting), done.

Searching through comments on PRs to find commits with a magic comment
string that haven’t been built by Jenkins is certainly what would be
lovely... but right now it would burn the GitHub rate limit (which is 5,000
requests per hour across the whole ASF) right through.



>
> Enrico
>
>
> > Other projects at ASF probably missed this point in the video series
> > chronicling the development of the plugin...
> >
> > On Sat 29 Dec 2018 at 13:10, Enrico Olivelli 
> wrote:
> >
> > > Hervè,
> > > This is the plugin
> > >
> > >
> >
> https://wiki.jenkins.io/display/JENKINS/GitHub+Branch+Source+Plugin?desktop=true=unmigrated-inline-wiki-markup
> > >
> > > I see our "maven-box" is using some special "Scan Apache Hosted Git
> > > Folder Triggers" source
> > > (https://builds.apache.org/job/maven-box/configure)
> > > In the Jenkins of my company in a "Multibranch Pipeline" I have a
> > > "Branch Sources" box with a "GitHub" option which lets me trigger
> > > builds by using PRs
> > >
> > >
> > > Enrico
> > >
> > > Il giorno sab 29 dic 2018 alle ore 13:43 Hervé BOUTEMY
> > >  ha scritto:
> > > >
> > > > Le samedi 29 décembre 2018, 12:40:20 CET Enrico Olivelli a écrit :
> > > > > Il sab 29 dic 2018, 12:37 Mickael Istria  ha
> > > scritto:
> > > > > > On Sat, Dec 29, 2018 at 12:01 PM Hervé BOUTEMY <
> > > herve.bout...@free.fr>
> > > > > >
> > > > > > wrote:
> > > > > > > But in both cases, currently, there is no automatic GitHub PR
> > > > > >
> > > > > > integration:
> > > > > > > Maven committers need to create a branch in the official
> > > repository to
> > > > > > > benefit
> > > > > > > from ASF Jenkins build
> > > > > >
> > > > > > Ah ok, I wasn't aware the GitHub repo was "unofficial" and
> couldn't
> > > be
> > > > > > used
> > > > > > to trigger builds. That sucks...
> > > > >
> > > > > Maven migrated to gitbox so actually github is an official repo for
> > > Maven.
> > > > > I see the same setup in Zookeeper and Bookkeeper and github pr
> plugin
> > > works
> > > > > like a charm (and I partecipated in setting it up)
> > > > oh great, that would be nice to have the same setup for Maven repos!
> > > >
> > > > >
> > > > > Enrico
> > > > >
> > > > > > Any idea how we could have GitHub PR reviews without this branch
> > > creation
> > > > > >
> > > > > > > by
> > > > > > > committers, be it at ASF or somewhere else?
> > > > > >
> > > > > > Using TravisCI could be a solution.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > -
> > > > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > > > For additional commands, e-mail: dev-h...@maven.apache.org
> > > >
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > > For additional commands, e-mail: dev-h...@maven.apache.org
> > >
> > > --
> > Sent from my phone
> >
> --
>
>
> -- Enrico Olivelli
>
-- 
Sent from my phone

Re: New Committers - community

[Enrico Olivelli]

Il sab 29 dic 2018, 15:17 Stephen Connolly 
ha scritto:

> There is a security issue with building PRs automatically.
>
> I can see about adding PR discovery to the existing ASF gitbox plugin, but
> we’d need committers to ok the build and have reviewed the code as the PR
> could contain attacks to be run from ASF hardware... which is why we don’t
> build PRs at present.
>

Now I have to review and then push to ASF repo and I have to tell to the
contributor that I will make CI start.
IMHO a good tradeoff is:
- a committer adds a 'test this please' comment, or '@asfbot test this
please' and then a CI job start
- this job updates the status line of the PR, with a link to the logs and
the status of the job

How does it sounds to you?

Enrico


> Other projects at ASF probably missed this point in the video series
> chronicling the development of the plugin...
>
> On Sat 29 Dec 2018 at 13:10, Enrico Olivelli  wrote:
>
> > Hervè,
> > This is the plugin
> >
> >
> https://wiki.jenkins.io/display/JENKINS/GitHub+Branch+Source+Plugin?desktop=true=unmigrated-inline-wiki-markup
> >
> > I see our "maven-box" is using some special "Scan Apache Hosted Git
> > Folder Triggers" source
> > (https://builds.apache.org/job/maven-box/configure)
> > In the Jenkins of my company in a "Multibranch Pipeline" I have a
> > "Branch Sources" box with a "GitHub" option which lets me trigger
> > builds by using PRs
> >
> >
> > Enrico
> >
> > Il giorno sab 29 dic 2018 alle ore 13:43 Hervé BOUTEMY
> >  ha scritto:
> > >
> > > Le samedi 29 décembre 2018, 12:40:20 CET Enrico Olivelli a écrit :
> > > > Il sab 29 dic 2018, 12:37 Mickael Istria  ha
> > scritto:
> > > > > On Sat, Dec 29, 2018 at 12:01 PM Hervé BOUTEMY <
> > herve.bout...@free.fr>
> > > > >
> > > > > wrote:
> > > > > > But in both cases, currently, there is no automatic GitHub PR
> > > > >
> > > > > integration:
> > > > > > Maven committers need to create a branch in the official
> > repository to
> > > > > > benefit
> > > > > > from ASF Jenkins build
> > > > >
> > > > > Ah ok, I wasn't aware the GitHub repo was "unofficial" and couldn't
> > be
> > > > > used
> > > > > to trigger builds. That sucks...
> > > >
> > > > Maven migrated to gitbox so actually github is an official repo for
> > Maven.
> > > > I see the same setup in Zookeeper and Bookkeeper and github pr plugin
> > works
> > > > like a charm (and I partecipated in setting it up)
> > > oh great, that would be nice to have the same setup for Maven repos!
> > >
> > > >
> > > > Enrico
> > > >
> > > > > Any idea how we could have GitHub PR reviews without this branch
> > creation
> > > > >
> > > > > > by
> > > > > > committers, be it at ASF or somewhere else?
> > > > >
> > > > > Using TravisCI could be a solution.
> > >
> > >
> > >
> > >
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > > For additional commands, e-mail: dev-h...@maven.apache.org
> > >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > For additional commands, e-mail: dev-h...@maven.apache.org
> >
> > --
> Sent from my phone
>
-- 


-- Enrico Olivelli

Re: New Committers - community

[Stephen Connolly]

There is a security issue with building PRs automatically.

I can see about adding PR discovery to the existing ASF gitbox plugin, but
we’d need committers to ok the build and have reviewed the code as the PR
could contain attacks to be run from ASF hardware... which is why we don’t
build PRs at present.

Other projects at ASF probably missed this point in the video series
chronicling the development of the plugin...

On Sat 29 Dec 2018 at 13:10, Enrico Olivelli  wrote:

> Hervè,
> This is the plugin
>
> https://wiki.jenkins.io/display/JENKINS/GitHub+Branch+Source+Plugin?desktop=true=unmigrated-inline-wiki-markup
>
> I see our "maven-box" is using some special "Scan Apache Hosted Git
> Folder Triggers" source
> (https://builds.apache.org/job/maven-box/configure)
> In the Jenkins of my company in a "Multibranch Pipeline" I have a
> "Branch Sources" box with a "GitHub" option which lets me trigger
> builds by using PRs
>
>
> Enrico
>
> Il giorno sab 29 dic 2018 alle ore 13:43 Hervé BOUTEMY
>  ha scritto:
> >
> > Le samedi 29 décembre 2018, 12:40:20 CET Enrico Olivelli a écrit :
> > > Il sab 29 dic 2018, 12:37 Mickael Istria  ha
> scritto:
> > > > On Sat, Dec 29, 2018 at 12:01 PM Hervé BOUTEMY <
> herve.bout...@free.fr>
> > > >
> > > > wrote:
> > > > > But in both cases, currently, there is no automatic GitHub PR
> > > >
> > > > integration:
> > > > > Maven committers need to create a branch in the official
> repository to
> > > > > benefit
> > > > > from ASF Jenkins build
> > > >
> > > > Ah ok, I wasn't aware the GitHub repo was "unofficial" and couldn't
> be
> > > > used
> > > > to trigger builds. That sucks...
> > >
> > > Maven migrated to gitbox so actually github is an official repo for
> Maven.
> > > I see the same setup in Zookeeper and Bookkeeper and github pr plugin
> works
> > > like a charm (and I partecipated in setting it up)
> > oh great, that would be nice to have the same setup for Maven repos!
> >
> > >
> > > Enrico
> > >
> > > > Any idea how we could have GitHub PR reviews without this branch
> creation
> > > >
> > > > > by
> > > > > committers, be it at ASF or somewhere else?
> > > >
> > > > Using TravisCI could be a solution.
> >
> >
> >
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > For additional commands, e-mail: dev-h...@maven.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> For additional commands, e-mail: dev-h...@maven.apache.org
>
> --
Sent from my phone

Re: New Committers - community

[Enrico Olivelli]

Hervè,
This is the plugin
https://wiki.jenkins.io/display/JENKINS/GitHub+Branch+Source+Plugin?desktop=true=unmigrated-inline-wiki-markup

I see our "maven-box" is using some special "Scan Apache Hosted Git
Folder Triggers" source
(https://builds.apache.org/job/maven-box/configure)
In the Jenkins of my company in a "Multibranch Pipeline" I have a
"Branch Sources" box with a "GitHub" option which lets me trigger
builds by using PRs


Enrico

Il giorno sab 29 dic 2018 alle ore 13:43 Hervé BOUTEMY
 ha scritto:
>
> Le samedi 29 décembre 2018, 12:40:20 CET Enrico Olivelli a écrit :
> > Il sab 29 dic 2018, 12:37 Mickael Istria  ha scritto:
> > > On Sat, Dec 29, 2018 at 12:01 PM Hervé BOUTEMY 
> > >
> > > wrote:
> > > > But in both cases, currently, there is no automatic GitHub PR
> > >
> > > integration:
> > > > Maven committers need to create a branch in the official repository to
> > > > benefit
> > > > from ASF Jenkins build
> > >
> > > Ah ok, I wasn't aware the GitHub repo was "unofficial" and couldn't be
> > > used
> > > to trigger builds. That sucks...
> >
> > Maven migrated to gitbox so actually github is an official repo for Maven.
> > I see the same setup in Zookeeper and Bookkeeper and github pr plugin works
> > like a charm (and I partecipated in setting it up)
> oh great, that would be nice to have the same setup for Maven repos!
>
> >
> > Enrico
> >
> > > Any idea how we could have GitHub PR reviews without this branch creation
> > >
> > > > by
> > > > committers, be it at ASF or somewhere else?
> > >
> > > Using TravisCI could be a solution.
>
>
>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> For additional commands, e-mail: dev-h...@maven.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: New Committers - community

[Hervé BOUTEMY]

Le samedi 29 décembre 2018, 12:40:20 CET Enrico Olivelli a écrit :
> Il sab 29 dic 2018, 12:37 Mickael Istria  ha scritto:
> > On Sat, Dec 29, 2018 at 12:01 PM Hervé BOUTEMY 
> > 
> > wrote:
> > > But in both cases, currently, there is no automatic GitHub PR
> > 
> > integration:
> > > Maven committers need to create a branch in the official repository to
> > > benefit
> > > from ASF Jenkins build
> > 
> > Ah ok, I wasn't aware the GitHub repo was "unofficial" and couldn't be
> > used
> > to trigger builds. That sucks...
> 
> Maven migrated to gitbox so actually github is an official repo for Maven.
> I see the same setup in Zookeeper and Bookkeeper and github pr plugin works
> like a charm (and I partecipated in setting it up)
oh great, that would be nice to have the same setup for Maven repos!

> 
> Enrico
> 
> > Any idea how we could have GitHub PR reviews without this branch creation
> > 
> > > by
> > > committers, be it at ASF or somewhere else?
> > 
> > Using TravisCI could be a solution.





-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: New Committers - community

[Hervé BOUTEMY]

Le samedi 29 décembre 2018, 12:37:36 CET Mickael Istria a écrit :
> On Sat, Dec 29, 2018 at 12:01 PM Hervé BOUTEMY 
> 
> wrote:
> > But in both cases, currently, there is no automatic GitHub PR integration:
> > Maven committers need to create a branch in the official repository to
> > benefit
> > from ASF Jenkins build
> 
> Ah ok, I wasn't aware the GitHub repo was "unofficial" and couldn't be used
> to trigger builds. That sucks...
GitHub repo is not unofficial: with GitBox, it's as official as ASF-hosted Git 
repo
it's GitHub PRs that are not real branches of the Git repo, then the 
triggerring has to be different than normal Git branches commits

> 
> Any idea how we could have GitHub PR reviews without this branch creation
> 
> > by
> > committers, be it at ASF or somewhere else?
> 
> Using TravisCI could be a solution.





-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: New Committers - community

[Enrico Olivelli]

Il sab 29 dic 2018, 12:37 Mickael Istria  ha scritto:

> On Sat, Dec 29, 2018 at 12:01 PM Hervé BOUTEMY 
> wrote:
>
> > But in both cases, currently, there is no automatic GitHub PR
> integration:
> > Maven committers need to create a branch in the official repository to
> > benefit
> > from ASF Jenkins build
> >
>
> Ah ok, I wasn't aware the GitHub repo was "unofficial" and couldn't be used
> to trigger builds. That sucks...
>

Maven migrated to gitbox so actually github is an official repo for Maven.
I see the same setup in Zookeeper and Bookkeeper and github pr plugin works
like a charm (and I partecipated in setting it up)

Enrico


> Any idea how we could have GitHub PR reviews without this branch creation
> > by
> > committers, be it at ASF or somewhere else?
> >
>
> Using TravisCI could be a solution.
>
-- 


-- Enrico Olivelli

Re: New Committers - community

[Mickael Istria]

On Sat, Dec 29, 2018 at 12:01 PM Hervé BOUTEMY 
wrote:

> But in both cases, currently, there is no automatic GitHub PR integration:
> Maven committers need to create a branch in the official repository to
> benefit
> from ASF Jenkins build
>

Ah ok, I wasn't aware the GitHub repo was "unofficial" and couldn't be used
to trigger builds. That sucks...

Any idea how we could have GitHub PR reviews without this branch creation
> by
> committers, be it at ASF or somewhere else?
>

Using TravisCI could be a solution.

Re: New Committers - community

[Enrico Olivelli]

Il sab 29 dic 2018, 12:19 Karl Heinz Marbaise  ha
scritto:

> Hi Mickael,
>
> On 29/12/18 09:57, Mickael Istria wrote:
> > On Mon, Dec 24, 2018 at 5:21 PM Karl Heinz Marbaise 
> > wrote:
> >
> >> Hi Mickael,
> >>
> >
> > Hi,
> >
> > I have to summarize this simply.
> >
> > I was confused about how automated tests are running or not. If I look at
> > https://github.com/apache/maven/pull/194 , I do not see any build/test
> > report on the issue. I imagine this requires someone to manually trigger
> > the tests, doesn't it?
> > If it is so, then it could be one step to remove on the review process,
> and
> > the automated build feedback would allow contributors to fix their PRs by
> > themselves before wasting time of a reviewer.
>
> I have started to discuss this with ASF Infra team ...
>
> Lets see if we find a solution...I see currently the issue in the amount
> of repos we have...but lets see...
>

We could use the github pr builder plugin but without automatic trigger, qe
can enable only the 'trigger phrase' so that is it enough for a commiter to
add a comment like 'test this please' in order to start the build and have
immediate feedback on github

Enrico


> Kind regards
> Karl Heinz Marb aise
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> For additional commands, e-mail: dev-h...@maven.apache.org
>
> --


-- Enrico Olivelli

Re: New Committers - community

[Karl Heinz Marbaise]

Hi Mickael,

On 29/12/18 09:57, Mickael Istria wrote:

On Mon, Dec 24, 2018 at 5:21 PM Karl Heinz Marbaise 
wrote:


Hi Mickael,



Hi,

I have to summarize this simply.

I was confused about how automated tests are running or not. If I look at
https://github.com/apache/maven/pull/194 , I do not see any build/test
report on the issue. I imagine this requires someone to manually trigger
the tests, doesn't it?
If it is so, then it could be one step to remove on the review process, and
the automated build feedback would allow contributors to fix their PRs by
themselves before wasting time of a reviewer.


I have started to discuss this with ASF Infra team ...

Lets see if we find a solution...I see currently the issue in the amount 
of repos we have...but lets see...


Kind regards
Karl Heinz Marb aise

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: New Committers - community

[Hervé BOUTEMY]

Le samedi 29 décembre 2018, 09:57:30 CET Mickael Istria a écrit :
> On Mon, Dec 24, 2018 at 5:21 PM Karl Heinz Marbaise 
> 
> wrote:
> > Hi Mickael,
> 
> Hi,
> 
> I have to summarize this simply.
> 
> > This is no lack of interest in any kind. It is simply a lack of time of
> > the committers...cause there is very limited number of committers...
> 
> I understand that, and I hope I was explicit enough when I was talking
> about the *perception* of contributions receiving low interest, as opposed
> to the reality you mention which is about lack of resources compared to
> backlog.
> In this case, some time ago, when Eclipse was in a similar position, we did
> collectively decide to make external contributions higher priority than our
> own work for a few months. This did have a very positive impact in growing
> the community, recruiting more developers, more committers, and also
> changing the mindset of most current committers to think more and more
> about incoming contributors and plan some time for the reviews, and
> prioritize community over development in some cases; because ultimately,
> the community makes the project alive more than its feature set.
> 
> > > It also seems like reviewing and testing PRs is not trivial, and that
> > 
> > more
> > 
> > > automation could help developers to trust incoming changes and deal with
> > > reviews.
> > 
> > What kind of ideas do you have? To be honest reviewing a PR takes time
> > and based on my experience no (other) automation will help there..but If
> > you have ideas please share them...
> 
> I was confused about how automated tests are running or not. If I look at
> https://github.com/apache/maven/pull/194 , I do not see any build/test
> report on the issue. I imagine this requires someone to manually trigger
> the tests, doesn't it?

see https://maven.apache.org/guides/development/guide-building-maven.html

for plugins and components, running the ITs is quite easy since it's only 
about activating "run-its" profile, there is no specific setup

for Maven core, it's more complex, because the its are in a separate git repo 
and multiple steps to run the ITs

But in both cases, currently, there is no automatic GitHub PR integration: 
Maven committers need to create a branch in the official repository to benefit 
from ASF Jenkins build

Any idea how we could have GitHub PR reviews without this branch creation by 
committers, be it at ASF or somewhere else?

> If it is so, then it could be one step to remove on the review process, and
> the automated build feedback would allow contributors to fix their PRs by
> themselves before wasting time of a reviewer.
> 
> > > This goes by having a mindset that makes core developers main task to
> > 
> > grow
> > 
> > > the community rather than fixing bugs or adding features.
> > 
> > That contradicts some of the feedback we got, cause we get feedback
> > saying why does it take so long to fix a bug etc...
> 
> I know that. It's also something we hear often in Eclipse and I guess
> something the majority of serious OSS projects face often.
> But it's OSS, people who need a fix should be ready to invest in it; and
> someone who's finding a bug long to fix is actually someone that can be
> turned as a contributor for their own bug ;)
> 
> The user community is very big but unfortunately the people who are
> 
> > willing to help is not that big...
> 
> Same as above ;)
> 
> > That said, I think Maven already enables some important success criteria,
> > 
> > > like being on GitHub. So I'm confident things can and will improve to
> > 
> > grow
> > 
> > > the community.
> > 
> > Over the time it already evolved/grown in several aspects. Maybe not in
> > the speed we wish it had...
> > But it takes time...
> 
> I have good hope it's worth it ;)





-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: New Committers - community

[Mickael Istria]

On Mon, Dec 24, 2018 at 5:21 PM Karl Heinz Marbaise 
wrote:

> Hi Mickael,
>

Hi,

I have to summarize this simply.
> This is no lack of interest in any kind. It is simply a lack of time of
> the committers...cause there is very limited number of committers...
>

I understand that, and I hope I was explicit enough when I was talking
about the *perception* of contributions receiving low interest, as opposed
to the reality you mention which is about lack of resources compared to
backlog.
In this case, some time ago, when Eclipse was in a similar position, we did
collectively decide to make external contributions higher priority than our
own work for a few months. This did have a very positive impact in growing
the community, recruiting more developers, more committers, and also
changing the mindset of most current committers to think more and more
about incoming contributors and plan some time for the reviews, and
prioritize community over development in some cases; because ultimately,
the community makes the project alive more than its feature set.



> > It also seems like reviewing and testing PRs is not trivial, and that
> more
> > automation could help developers to trust incoming changes and deal with
> > reviews.
>
> What kind of ideas do you have? To be honest reviewing a PR takes time
> and based on my experience no (other) automation will help there..but If
> you have ideas please share them...
>

I was confused about how automated tests are running or not. If I look at
https://github.com/apache/maven/pull/194 , I do not see any build/test
report on the issue. I imagine this requires someone to manually trigger
the tests, doesn't it?
If it is so, then it could be one step to remove on the review process, and
the automated build feedback would allow contributors to fix their PRs by
themselves before wasting time of a reviewer.


> > This goes by having a mindset that makes core developers main task to
> grow
> > the community rather than fixing bugs or adding features.
>
> That contradicts some of the feedback we got, cause we get feedback
> saying why does it take so long to fix a bug etc...
>

I know that. It's also something we hear often in Eclipse and I guess
something the majority of serious OSS projects face often.
But it's OSS, people who need a fix should be ready to invest in it; and
someone who's finding a bug long to fix is actually someone that can be
turned as a contributor for their own bug ;)

The user community is very big but unfortunately the people who are
> willing to help is not that big...
>

Same as above ;)

> That said, I think Maven already enables some important success criteria,
> > like being on GitHub. So I'm confident things can and will improve to
> grow
> > the community.
>
> Over the time it already evolved/grown in several aspects. Maybe not in
> the speed we wish it had...
> But it takes time...
>

I have good hope it's worth it ;)
-- 
Mickael Istria
Eclipse IDE 
developer, for Red Hat Developers 

Re: New Committers - community

[Michael Osipov]

Am 2018-12-24 um 17:21 schrieb Karl Heinz Marbaise:

Hi Mickael,

I have separated this thread from the previous one cause this will mixup 
things...


On 24/12/18 15:40, Mickael Istria wrote:

Hi,

On Sunday, December 23, 2018, Stephen Connolly <
stephen.alan.conno...@gmail.com> wrote:


If we have good reliable tooling, then we hope to be able to encourage
contributions (as you don’t need a CLA for an “obvious” ~10-15 line 
patch -

intent to submit is sufficient) and get the code base more energised.



As a tentative new contributor, I'd like to voice that it seems to me 
that

the issues in getting more contributors are not first about code style or
technologies. (This is based on my experience.in the Eclipse IDE project
where some components were in a bad shape in term of community activity,
comparable to how Maven is right now, and managed to get better.)




While cleaner more modern and more «inclusive» code helps, adapting to
another code style or older code is still a simple enough exercise for
contributors to keep motivated and providing code.


What I feel missing in Maven project to make contributors feel more 
welcome
is reactivity and guidance of the core developers towards 
contributions: it

takes a lot of time to get feedback on a Jira and a PR, and I had to ping
contributors to get feedback. I still don't get why my patches are not 
yet
merged while feedback seems good. 


 > Being that insisting isn't something many
people can afford or like or dare to do. This lack of apparent 
interest in

external contributions is what can kill any open-source project.


I have to summarize this simply.

This is no lack of interest in any kind. It is simply a lack of time of 
the committers...cause there is very limited number of committers...


This is something I exprience myself, in the last couple of months I 
bare had time to do anything for Maven. I do not intend to briskly pass 
an PR. Many PRs require deep understanding of the code as well as the 
issue and the proposed fixed. It may take several days to assess that.


It'd be great if we could have at least one paid dev just like Mark 
Thomas for Tomcat.


It also seems like reviewing and testing PRs is not trivial, and that 
more

automation could help developers to trust incoming changes and deal with
reviews.


We already made a large steps into the right direction but this does not 
mean to make more of them...


What kind of ideas do you have? To be honest reviewing a PR takes time 
and based on my experience no (other) automation will help there..but If 
you have ideas please share them...



So it you want more contributors in Maven,I think the #1 item by far 
is to

make sure PRs are reviewed and merged promptly and then that the «time to
market» between a patch and a release is short enough to not leave 
time to

contributors to consider competitve technologies or workarounds they'll
never contribute back.


This goes by having a mindset that makes core developers main task to 
grow

the community rather than fixing bugs or adding features.


That contradicts some of the feedback we got, cause we get feedback 
saying why does it take so long to fix a bug etc...


But of course I do understand your point which is the right direction to 
go...





About Maven 4.0, I don't have an opinion and am not enthusiast (that's my
natural reaction to revolutions over simple evolution). I'm curious about
what's the added value to produce and whether it's worth the risk and
effort.


But again, from my experience with Eclipse IDE,I think it's 
fundamental to

get a vibrant and flawless developer community before starting such a
revolution; and I don't have the impression current Maven community is 
big

and agile enough to sustain big changes.


The user community is very big but unfortunately the people who are 
willing to help is not that big...


Absolutely, it pretty much seems that people take Maven for granted 
because there are some other "idiots" who will solve my build problem.
I can give an example, my employer has 300 000+ employees, thousands of 
devs, a lot of them are using Maven and other ASF stuff, I am one of a 
few who are contributing to ASF code. That's pretty weak. I guess this 
applies to most companies from the "old" world.


Michael


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org