Re: [Proposal] Automation For Beam Dependency Check

[Kenneth Knowles]

This kind of leaking analysis that `mvn dependency:analyze` does is I think
what is also called IWYU (Include What You Use). I looked around and there
are some gradle plugins to do the same thing. I couldn't tell which was the
most robust.

Kenn

On Mon, Jun 4, 2018 at 9:46 AM Chamikara Jayalath 
wrote:

>
>
> On Mon, Jun 4, 2018 at 6:10 AM Ismaël Mejía  wrote:
>
>> Is there a way to add to that weekly report the new dependencies that
>> were introduced in the week before, or that have changed?
>>
>
> I think it makes sense to add a recent changes section so that community
> is up to date and can discuss if there are any possible issues. For
> example, (1) new dependencies with known critical vulnerabilities (2)
> component level dependency version overrides that can be avoided.
>
>
>>
>> We are not addressing another important problem: Leaking of
>> dependencies. I am not aware of the gradle equivalent of the maven
>> dependency plugin that helps to determine missing dependencies (non
>> explicitly defined) or unused dependencies. Is there any way to
>> achieve this too? (Note this should probably be enforced at Jenkins
>> not part of the report but just curious)
>>
>
> Agree that this probably be enforced through possibly PreCommit Jenkins
> job instead of the job proposed here.
>
> Regarding leaking, did you mean cross-component leaks (one Beam component
> leaking a dependency to another Beam component) or something else ? For
> cross-component dependency leaks, following proposal promotes using
> versions defined at the top level which will help avoid this issue.
>
> https://docs.google.com/document/d/15m1MziZ5TNd9rh_XN0YYBJfYkt0Oj-Ou9g0KFDPL2aA/edit?usp=sharing
>
> Thanks,
> Cham
>
>
>>
>> On Wed, May 30, 2018 at 5:16 AM Yifan Zou  wrote:
>> >
>> > Thanks everyone for making comments and suggestions. I modified the
>> proposal that added dependency release time as the major criteria for
>> outdated package determination.
>> > The revised doc is here:
>> https://docs.google.com/document/d/1rqr_8a9NYZCgeiXpTIwWLCL7X8amPAVfRXsO72BpBwA.
>> Any comments are welcome.
>> >
>> > -Yifan
>> >
>> > On Thu, May 24, 2018 at 5:25 PM Chamikara Jayalath <
>> chamik...@google.com> wrote:
>> >>
>> >> Thanks Yifan. Added some comments. I think having regularly generated
>> human reports on outdated decencies of Beam SDKs will be extremely helpful
>> in keeping Beam in a healthy state.
>> >>
>> >> - Cham
>> >>
>> >> On Thu, May 24, 2018 at 7:08 AM Yifan Zou  wrote:
>> >>>
>> >>> Hello,
>> >>>
>> >>> I have a proposal to automate Beam dependency check. Since some Beam
>> dependent packages are out-of-date, we want to identify them and check for
>> dependency updates regularly in the future. Generally, we have couple
>> options to do it:
>> >>> 1. Implementing a Jenkins job that check dependency versions and
>> create reports.
>> >>> 2. Using the Github App Dependabot to automate dependency updates.
>> >>> 3. Combination of those two solutions.
>> >>>
>> >>> I am looking forward to hearing feedback from you :)
>> >>>
>> >>>
>> https://docs.google.com/document/d/1rqr_8a9NYZCgeiXpTIwWLCL7X8amPAVfRXsO72BpBwA/
>> >>>
>> >>> Thanks.
>> >>>
>> >>> Best.
>> >>> Yifan Zou
>>
>

Re: [Proposal] Automation For Beam Dependency Check

[Chamikara Jayalath]

On Mon, Jun 4, 2018 at 6:10 AM Ismaël Mejía  wrote:

> Is there a way to add to that weekly report the new dependencies that
> were introduced in the week before, or that have changed?
>

I think it makes sense to add a recent changes section so that community is
up to date and can discuss if there are any possible issues. For example,
(1) new dependencies with known critical vulnerabilities (2) component
level dependency version overrides that can be avoided.


>
> We are not addressing another important problem: Leaking of
> dependencies. I am not aware of the gradle equivalent of the maven
> dependency plugin that helps to determine missing dependencies (non
> explicitly defined) or unused dependencies. Is there any way to
> achieve this too? (Note this should probably be enforced at Jenkins
> not part of the report but just curious)
>

Agree that this probably be enforced through possibly PreCommit Jenkins job
instead of the job proposed here.

Regarding leaking, did you mean cross-component leaks (one Beam component
leaking a dependency to another Beam component) or something else ? For
cross-component dependency leaks, following proposal promotes using
versions defined at the top level which will help avoid this issue.
https://docs.google.com/document/d/15m1MziZ5TNd9rh_XN0YYBJfYkt0Oj-Ou9g0KFDPL2aA/edit?usp=sharing

Thanks,
Cham


>
> On Wed, May 30, 2018 at 5:16 AM Yifan Zou  wrote:
> >
> > Thanks everyone for making comments and suggestions. I modified the
> proposal that added dependency release time as the major criteria for
> outdated package determination.
> > The revised doc is here:
> https://docs.google.com/document/d/1rqr_8a9NYZCgeiXpTIwWLCL7X8amPAVfRXsO72BpBwA.
> Any comments are welcome.
> >
> > -Yifan
> >
> > On Thu, May 24, 2018 at 5:25 PM Chamikara Jayalath 
> wrote:
> >>
> >> Thanks Yifan. Added some comments. I think having regularly generated
> human reports on outdated decencies of Beam SDKs will be extremely helpful
> in keeping Beam in a healthy state.
> >>
> >> - Cham
> >>
> >> On Thu, May 24, 2018 at 7:08 AM Yifan Zou  wrote:
> >>>
> >>> Hello,
> >>>
> >>> I have a proposal to automate Beam dependency check. Since some Beam
> dependent packages are out-of-date, we want to identify them and check for
> dependency updates regularly in the future. Generally, we have couple
> options to do it:
> >>> 1. Implementing a Jenkins job that check dependency versions and
> create reports.
> >>> 2. Using the Github App Dependabot to automate dependency updates.
> >>> 3. Combination of those two solutions.
> >>>
> >>> I am looking forward to hearing feedback from you :)
> >>>
> >>>
> https://docs.google.com/document/d/1rqr_8a9NYZCgeiXpTIwWLCL7X8amPAVfRXsO72BpBwA/
> >>>
> >>> Thanks.
> >>>
> >>> Best.
> >>> Yifan Zou
>

Re: [Proposal] Automation For Beam Dependency Check

[Ismaël Mejía]

Is there a way to add to that weekly report the new dependencies that
were introduced in the week before, or that have changed?

We are not addressing another important problem: Leaking of
dependencies. I am not aware of the gradle equivalent of the maven
dependency plugin that helps to determine missing dependencies (non
explicitly defined) or unused dependencies. Is there any way to
achieve this too? (Note this should probably be enforced at Jenkins
not part of the report but just curious)

On Wed, May 30, 2018 at 5:16 AM Yifan Zou  wrote:
>
> Thanks everyone for making comments and suggestions. I modified the proposal 
> that added dependency release time as the major criteria for outdated package 
> determination.
> The revised doc is here: 
> https://docs.google.com/document/d/1rqr_8a9NYZCgeiXpTIwWLCL7X8amPAVfRXsO72BpBwA.
>  Any comments are welcome.
>
> -Yifan
>
> On Thu, May 24, 2018 at 5:25 PM Chamikara Jayalath  
> wrote:
>>
>> Thanks Yifan. Added some comments. I think having regularly generated human 
>> reports on outdated decencies of Beam SDKs will be extremely helpful in 
>> keeping Beam in a healthy state.
>>
>> - Cham
>>
>> On Thu, May 24, 2018 at 7:08 AM Yifan Zou  wrote:
>>>
>>> Hello,
>>>
>>> I have a proposal to automate Beam dependency check. Since some Beam 
>>> dependent packages are out-of-date, we want to identify them and check for 
>>> dependency updates regularly in the future. Generally, we have couple 
>>> options to do it:
>>> 1. Implementing a Jenkins job that check dependency versions and create 
>>> reports.
>>> 2. Using the Github App Dependabot to automate dependency updates.
>>> 3. Combination of those two solutions.
>>>
>>> I am looking forward to hearing feedback from you :)
>>>
>>> https://docs.google.com/document/d/1rqr_8a9NYZCgeiXpTIwWLCL7X8amPAVfRXsO72BpBwA/
>>>
>>> Thanks.
>>>
>>> Best.
>>> Yifan Zou

Re: [Proposal] Automation For Beam Dependency Check

[Yifan Zou]

Thanks everyone for making comments and suggestions. I modified the
proposal that added dependency release time as the major criteria for
outdated package determination.
The revised doc is here:
https://docs.google.com/document/d/1rqr_8a9NYZCgeiXpTIwWLCL7X8amPAVfRXsO72BpBwA.
Any comments are welcome.

-Yifan

On Thu, May 24, 2018 at 5:25 PM Chamikara Jayalath 
wrote:

> Thanks Yifan. Added some comments. I think having regularly generated
> human reports on outdated decencies of Beam SDKs will be extremely helpful
> in keeping Beam in a healthy state.
>
> - Cham
>
> On Thu, May 24, 2018 at 7:08 AM Yifan Zou  wrote:
>
>> Hello,
>>
>> I have a proposal to automate Beam dependency check. Since some Beam
>> dependent packages are out-of-date, we want to identify them and check for
>> dependency updates regularly in the future. Generally, we have couple
>> options to do it:
>> 1. Implementing a Jenkins job that check dependency versions and
>> create reports.
>> 2. Using the Github App Dependabot
>>  to automate dependency
>> updates.
>> 3. Combination of those two solutions.
>>
>> I am looking forward to hearing feedback from you :)
>>
>>
>> https://docs.google.com/document/d/1rqr_8a9NYZCgeiXpTIwWLCL7X8amPAVfRXsO72BpBwA/
>>
>> Thanks.
>>
>> Best.
>> Yifan Zou
>>
>

Re: [Proposal] Automation For Beam Dependency Check

[Chamikara Jayalath]

Thanks Yifan. Added some comments. I think having regularly generated human
reports on outdated decencies of Beam SDKs will be extremely helpful in
keeping Beam in a healthy state.

- Cham

On Thu, May 24, 2018 at 7:08 AM Yifan Zou  wrote:

> Hello,
>
> I have a proposal to automate Beam dependency check. Since some Beam
> dependent packages are out-of-date, we want to identify them and check for
> dependency updates regularly in the future. Generally, we have couple
> options to do it:
> 1. Implementing a Jenkins job that check dependency versions and
> create reports.
> 2. Using the Github App Dependabot
>  to automate dependency
> updates.
> 3. Combination of those two solutions.
>
> I am looking forward to hearing feedback from you :)
>
>
> https://docs.google.com/document/d/1rqr_8a9NYZCgeiXpTIwWLCL7X8amPAVfRXsO72BpBwA/
>
> Thanks.
>
> Best.
> Yifan Zou
>