Re: Running source code analysis on Try

[Bastien Abadie]

Hello Nicholas,

First, i'm sorry the bot did not analyze your patch. I've asked a
Phabricator admin to restart the build, and it went fine
.

I'm aware of the issue
 you reported and
I am actively working on it. A fix should be in production in the next few
days.

I don't want to bore you with the details, but your build failed on the
first step for the code review. This is the only part we are hosting on
Heroku, and that does not scale easily. We are in the process of moving
that piece to GCP with CloudOps
; things should be
better there.

Finally, as any distributed or complex system, an SLA of 100% is impossible
to achieve. If you encounter an issue later on, feel free to contact me
directly, or post on the Slack channel #phabricator

Cheers,

Bastien Abadie

On Thu, May 16, 2019 at 6:18 AM Nicholas Alexander 
wrote:

> Bastien, others,
>
> On Tue, May 7, 2019 at 9:38 AM Bastien Abadie  wrote:
>
>> TL;DR: We are leveraging the try infrastructure to perform source code
>> analysis (linters, static analyzers, etc). You can take advantage of this
>> to trigger other try jobs on Phabricator revisions.
>>
>
> What is the status of this work?  I want to take advantage of the new
> infrastructure to finally turn certain Android analysis tasks into proper
> lints (see Bug 1512487 [1]) but the "Plan 4 Source Code Analysis" build
> plan appears to both instantly fail and be opaque (see, for example [2]).
> I believe that "reviewbot" was very unhappy last night; perhaps that is
> related?
>
> Even with setbacks, I'm excited for these changes to make it easier to
> build analysis tasks!
> Nick
>
> [1] https://bugzilla.mozilla.org/show_bug.cgi?id=1512487
> [2] https://phabricator.services.mozilla.com/harbormaster/build/89310/
>
>
>>
>> For more than a year, the Release Management & Quality team has been
>> running the Code review bot
>>  using several
>> Mozilla in-house projects: Taskcluster  &
>> release-services .
>>
>> Our process was simple at first: for every new patch on Mozreview, then
>> Phabricator, we would run a few analyzers in a Taskcluster task. It worked
>> well for some time, but did not provide the flexibility nor the
>> performance
>> needed to support more analyzers (clang-format, Infer, Coverity, …)
>>
>> So we decided to experiment with a more scalable architecture, using
>> Treeherder/Try to run all necessary analyzers for your patches, in
>> parallel.
>>
>> This move will give us a lot of benefits:
>>
>>-
>>
>>The analyzers themselves are well defined and exposed in
>>mozilla-central, allowing us to use the exact same tools as our usual
>> CI
>>and the developers on their computers.
>>-
>>
>>Better overall performance and stability (from mercurial clones to
>>running analyzers safely in parallel)
>>-
>>
>>A standard analyzer output in JSON will allow us to easily add new
>>analyzers. These new implementations could even come from you!
>>
>>
>> Over the last few months, we have been building the different pieces
>> needed
>> to move the bot on the Try infrastructure, and today, I’m glad to announce
>> it’s running in production, on your revisions!
>>
>> Any new Diff on Phabricator should now have a build plan named Source Code
>> Analysis .
>> Phabricator triggers a code review as soon as the diff is published,
>> automatically creating a new try job. You’ll get a “Treeherder Jobs” link
>> next to the build plan when it’s created by the bot, allowing you to check
>> on the analysis progress, and dive into the issues.
>>
>>
>> You will also be able to use Treeherder to add new jobs, effectively
>> making
>> it possible to run try tests for Phabricator revisions in just a few
>> clicks. On a Treeherder push, open its action menu on the right side
>> (right
>> after the Pin button). The "Add new jobs" options shows all the available
>> jobs grouped by platform and test suite, and you can click to select and
>> then submit. "Add new jobs (Search)" allows you to search in the job names
>> like mach try fuzzy and add them.
>>
>> If the patch contains any issues, they will be reported as before, through
>> inline comments. You can now restart a build if it fails (click on the
>> Build, there is a “Restart Build” link in the right sidebar). Please note
>> that secure revisions are not supported for now, but we are actively
>> working on that (a build failure will be shown for secure revisions for
>> the
>> time being).
>>
>> We would like to thank the different people and teams involved in this
>> effort: Andrew from the Automation team; Tom, Rail, Chris and Rok from
>> Release Engineering; 

Re: Running source code analysis on Try

[Nicholas Alexander]

Bastien, others,

On Tue, May 7, 2019 at 9:38 AM Bastien Abadie  wrote:

> TL;DR: We are leveraging the try infrastructure to perform source code
> analysis (linters, static analyzers, etc). You can take advantage of this
> to trigger other try jobs on Phabricator revisions.
>

What is the status of this work?  I want to take advantage of the new
infrastructure to finally turn certain Android analysis tasks into proper
lints (see Bug 1512487 [1]) but the "Plan 4 Source Code Analysis" build
plan appears to both instantly fail and be opaque (see, for example [2]).
I believe that "reviewbot" was very unhappy last night; perhaps that is
related?

Even with setbacks, I'm excited for these changes to make it easier to
build analysis tasks!
Nick

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1512487
[2] https://phabricator.services.mozilla.com/harbormaster/build/89310/


>
> For more than a year, the Release Management & Quality team has been
> running the Code review bot
>  using several
> Mozilla in-house projects: Taskcluster  &
> release-services .
>
> Our process was simple at first: for every new patch on Mozreview, then
> Phabricator, we would run a few analyzers in a Taskcluster task. It worked
> well for some time, but did not provide the flexibility nor the performance
> needed to support more analyzers (clang-format, Infer, Coverity, …)
>
> So we decided to experiment with a more scalable architecture, using
> Treeherder/Try to run all necessary analyzers for your patches, in
> parallel.
>
> This move will give us a lot of benefits:
>
>-
>
>The analyzers themselves are well defined and exposed in
>mozilla-central, allowing us to use the exact same tools as our usual CI
>and the developers on their computers.
>-
>
>Better overall performance and stability (from mercurial clones to
>running analyzers safely in parallel)
>-
>
>A standard analyzer output in JSON will allow us to easily add new
>analyzers. These new implementations could even come from you!
>
>
> Over the last few months, we have been building the different pieces needed
> to move the bot on the Try infrastructure, and today, I’m glad to announce
> it’s running in production, on your revisions!
>
> Any new Diff on Phabricator should now have a build plan named Source Code
> Analysis .
> Phabricator triggers a code review as soon as the diff is published,
> automatically creating a new try job. You’ll get a “Treeherder Jobs” link
> next to the build plan when it’s created by the bot, allowing you to check
> on the analysis progress, and dive into the issues.
>
>
> You will also be able to use Treeherder to add new jobs, effectively making
> it possible to run try tests for Phabricator revisions in just a few
> clicks. On a Treeherder push, open its action menu on the right side (right
> after the Pin button). The "Add new jobs" options shows all the available
> jobs grouped by platform and test suite, and you can click to select and
> then submit. "Add new jobs (Search)" allows you to search in the job names
> like mach try fuzzy and add them.
>
> If the patch contains any issues, they will be reported as before, through
> inline comments. You can now restart a build if it fails (click on the
> Build, there is a “Restart Build” link in the right sidebar). Please note
> that secure revisions are not supported for now, but we are actively
> working on that (a build failure will be shown for secure revisions for the
> time being).
>
> We would like to thank the different people and teams involved in this
> effort: Andrew from the Automation team; Tom, Rail, Chris and Rok from
> Release Engineering; Dustin and Peter from the Taskcluster team; Steven,
> glob and David from the Engineering Workflow team.
>
> If you have any questions regarding this move or the code review bot, feel
> free to contact us by mail or on IRC #static-analyzers.
>
> Bastien, on the behalf of the Release Management and Quality team.
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Running source code analysis on Try

[Bastien Abadie]

TL;DR: We are leveraging the try infrastructure to perform source code
analysis (linters, static analyzers, etc). You can take advantage of this
to trigger other try jobs on Phabricator revisions.

For more than a year, the Release Management & Quality team has been
running the Code review bot
 using several
Mozilla in-house projects: Taskcluster  &
release-services .

Our process was simple at first: for every new patch on Mozreview, then
Phabricator, we would run a few analyzers in a Taskcluster task. It worked
well for some time, but did not provide the flexibility nor the performance
needed to support more analyzers (clang-format, Infer, Coverity, …)

So we decided to experiment with a more scalable architecture, using
Treeherder/Try to run all necessary analyzers for your patches, in parallel.

This move will give us a lot of benefits:

   -

   The analyzers themselves are well defined and exposed in
   mozilla-central, allowing us to use the exact same tools as our usual CI
   and the developers on their computers.
   -

   Better overall performance and stability (from mercurial clones to
   running analyzers safely in parallel)
   -

   A standard analyzer output in JSON will allow us to easily add new
   analyzers. These new implementations could even come from you!


Over the last few months, we have been building the different pieces needed
to move the bot on the Try infrastructure, and today, I’m glad to announce
it’s running in production, on your revisions!

Any new Diff on Phabricator should now have a build plan named Source Code
Analysis .
Phabricator triggers a code review as soon as the diff is published,
automatically creating a new try job. You’ll get a “Treeherder Jobs” link
next to the build plan when it’s created by the bot, allowing you to check
on the analysis progress, and dive into the issues.


You will also be able to use Treeherder to add new jobs, effectively making
it possible to run try tests for Phabricator revisions in just a few
clicks. On a Treeherder push, open its action menu on the right side (right
after the Pin button). The "Add new jobs" options shows all the available
jobs grouped by platform and test suite, and you can click to select and
then submit. "Add new jobs (Search)" allows you to search in the job names
like mach try fuzzy and add them.

If the patch contains any issues, they will be reported as before, through
inline comments. You can now restart a build if it fails (click on the
Build, there is a “Restart Build” link in the right sidebar). Please note
that secure revisions are not supported for now, but we are actively
working on that (a build failure will be shown for secure revisions for the
time being).

We would like to thank the different people and teams involved in this
effort: Andrew from the Automation team; Tom, Rail, Chris and Rok from
Release Engineering; Dustin and Peter from the Taskcluster team; Steven,
glob and David from the Engineering Workflow team.

If you have any questions regarding this move or the code review bot, feel
free to contact us by mail or on IRC #static-analyzers.

Bastien, on the behalf of the Release Management and Quality team.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform