Re: Progress with automating testing of patches

[Joshua Branson]

Ludovic Courtès  writes:

> Hi,
>
> jbra...@dismail.de skribis:
>
>> I just created a debbugs-guix.el file in debbugs.  The update should be
>> available on elpa by now.
>>
>> https://issues.guix.gnu.org/56987
>
> Nice!  So that’ll be part of the next ‘emacs-debbugs’ package, right?
>

Yes sir!  It provides debbugs-gnu-guix-search 
debbugs-gnu-my-open-bugs functions.  

>
> Ludo’.

Re: Progress with automating testing of patches

[Ludovic Courtès]

Hi,

jbra...@dismail.de skribis:

> I just created a debbugs-guix.el file in debbugs.  The update should be
> available on elpa by now.
>
> https://issues.guix.gnu.org/56987

Nice!  So that’ll be part of the next ‘emacs-debbugs’ package, right?

Ludo’.

Re: Progress with automating testing of patches

[Ludovic Courtès]

Hi,

Christopher Baines  skribis:

> data.guix.gnu.org just tracks the master branch, and doesn't delete any
> data.
>
> data.qa.guix.gnu.org tracks all branches in the main Guix repository,
> plus branches for issues, and regularly deletes revisions where they're
> either old or the branch no longer exists.
>
> There's nothing technically preventing just using one instance for both
> purposes, it's just operationally I thought it would be easier to split
> the concerns across two instances.

Got it, that makes a lot of sense to me.  Thanks!

Ludo’.

Re: Progress with automating testing of patches

[jbranso]

October 1, 2022 1:08 PM, "Ludovic Courtès"  wrote:

> Hello!
> 
> As discussed in Paris, I’m a big fan of qa.guix.gnu.org! I like that it
> shows all the information relevant to packagers and reviewers in a
> concise way.
> 
> I wonder if it’s due to recent changes since I last looked, but I’m a
> bit confused by the numbers in this example:
> 
> https://qa.guix.gnu.org/issue/58186
> 
> The numbers before/after patches don’t match and the lint warnings (nice
> addition!) appear to unrelated to the patch at hand.
> 
> Any idea what’s going on?
> 
> Conversely,  looks fine, for
> example.
> 
> BTW, Emacs users, I have this key binding that I find useful:
> 
> --8<---cut here---start->8---
> (defun ludo-jump-to-guix-qa-url ()
> "Jump to the QA page of the Debbugs issue at point."
> (interactive)
> (let ((url (concat "https://qa.guix.gnu.org/issue;
> (number-to-string (debbugs-gnu-current-id)
> (browse-url url)))
> 
> (define-key debbugs-gnu-mode-map (kbd "C-M-j") 'ludo-jump-to-guix-qa-url)
> --8<---cut here---end--->8---

Would it make sense to add something like this to debbugs?

I just created a debbugs-guix.el file in debbugs.  The update should be
available on elpa by now.

https://issues.guix.gnu.org/56987

> 
> Thanks!
> 
> Ludo’.

Re: Progress with automating testing of patches

[Christopher Baines]

Ludovic Courtès  writes:

> Hi,
>
> Christopher Baines  skribis:
>
>> Ludovic Courtès  writes:
>>
>>> I wonder if it’s due to recent changes since I last looked, but I’m a
>>> bit confused by the numbers in this example:
>>>
>>>   https://qa.guix.gnu.org/issue/58186
>>>
>>> The numbers before/after patches don’t match and the lint warnings (nice
>>> addition!) appear to unrelated to the patch at hand.
>>>
>>> Any idea what’s going on?
>>
>> I've had an initial look. One important clue is that the basis of the
>> comparison [1] differs between data.qa.guix.gnu.org and
>> data.guix.gnu.org [2]. The package number differs for example.
>>
>> 1: 
>> https://data.qa.guix.gnu.org/revision/e6777cfa5eb5e9c36eaf7810b42cac0fbcaa367c
>> 2: 
>> https://data.guix.gnu.org/revision/e6777cfa5eb5e9c36eaf7810b42cac0fbcaa367c
>>
>> That shouldn't happen, one revision of Guix should have the same number
>> of packages regardless of what server looked at it. This being wrong
>> explains the bad data on qa.guix.gnu.org, since the comparison being
>> done by data.qa.guix.gnu.org is based on bad data.
>
> OK, I see.
>
> BTW, why do we have both data.qa.guix and data.guix?  These are both
> instances of the same service, right?

Yep, but the configuration is different.

data.guix.gnu.org just tracks the master branch, and doesn't delete any
data.

data.qa.guix.gnu.org tracks all branches in the main Guix repository,
plus branches for issues, and regularly deletes revisions where they're
either old or the branch no longer exists.

There's nothing technically preventing just using one instance for both
purposes, it's just operationally I thought it would be easier to split
the concerns across two instances.


signature.asc
Description: PGP signature

Re: Progress with automating testing of patches

[Ludovic Courtès]

Hi,

Christopher Baines  skribis:

> Ludovic Courtès  writes:
>
>> I wonder if it’s due to recent changes since I last looked, but I’m a
>> bit confused by the numbers in this example:
>>
>>   https://qa.guix.gnu.org/issue/58186
>>
>> The numbers before/after patches don’t match and the lint warnings (nice
>> addition!) appear to unrelated to the patch at hand.
>>
>> Any idea what’s going on?
>
> I've had an initial look. One important clue is that the basis of the
> comparison [1] differs between data.qa.guix.gnu.org and
> data.guix.gnu.org [2]. The package number differs for example.
>
> 1: 
> https://data.qa.guix.gnu.org/revision/e6777cfa5eb5e9c36eaf7810b42cac0fbcaa367c
> 2: https://data.guix.gnu.org/revision/e6777cfa5eb5e9c36eaf7810b42cac0fbcaa367c
>
> That shouldn't happen, one revision of Guix should have the same number
> of packages regardless of what server looked at it. This being wrong
> explains the bad data on qa.guix.gnu.org, since the comparison being
> done by data.qa.guix.gnu.org is based on bad data.

OK, I see.

BTW, why do we have both data.qa.guix and data.guix?  These are both
instances of the same service, right?

Thanks,
Ludo’.

Re: Progress with automating testing of patches

[Christopher Baines]

Ludovic Courtès  writes:

> I wonder if it’s due to recent changes since I last looked, but I’m a
> bit confused by the numbers in this example:
>
>   https://qa.guix.gnu.org/issue/58186
>
> The numbers before/after patches don’t match and the lint warnings (nice
> addition!) appear to unrelated to the patch at hand.
>
> Any idea what’s going on?

I've had an initial look. One important clue is that the basis of the
comparison [1] differs between data.qa.guix.gnu.org and
data.guix.gnu.org [2]. The package number differs for example.

1: 
https://data.qa.guix.gnu.org/revision/e6777cfa5eb5e9c36eaf7810b42cac0fbcaa367c
2: https://data.guix.gnu.org/revision/e6777cfa5eb5e9c36eaf7810b42cac0fbcaa367c

That shouldn't happen, one revision of Guix should have the same number
of packages regardless of what server looked at it. This being wrong
explains the bad data on qa.guix.gnu.org, since the comparison being
done by data.qa.guix.gnu.org is based on bad data.

This reminds me of [3] where data.guix.gnu.org processed a revision and
somehow got things wrong.

3: https://lists.gnu.org/archive/html/guix-devel/2021-09/msg00192.html

I'll try investigating this further when I have more time, there should
be locking in place so that even when multiple jobs are being processed
at the same time, only one job at a time is able to call
latest-channel-instances, so I don't currently have a theory as to how
this goes wrong. I think I added more logging off the back of the
previous issue, so maybe I'll be able to get further this time.

Thanks,

Chris


signature.asc
Description: PGP signature

Re: Progress with automating testing of patches

[Ludovic Courtès]

Hello!

As discussed in Paris, I’m a big fan of qa.guix.gnu.org!  I like that it
shows all the information relevant to packagers and reviewers in a
concise way.

I wonder if it’s due to recent changes since I last looked, but I’m a
bit confused by the numbers in this example:

  https://qa.guix.gnu.org/issue/58186

The numbers before/after patches don’t match and the lint warnings (nice
addition!) appear to unrelated to the patch at hand.

Any idea what’s going on?

Conversely,  looks fine, for
example.

BTW, Emacs users, I have this key binding that I find useful:

--8<---cut here---start->8---
(defun ludo-jump-to-guix-qa-url ()
  "Jump to the QA page of the Debbugs issue at point."
  (interactive)
  (let ((url (concat "https://qa.guix.gnu.org/issue/;
 (number-to-string (debbugs-gnu-current-id)
(browse-url url)))

(define-key debbugs-gnu-mode-map (kbd "C-M-j") 'ludo-jump-to-guix-qa-url)
--8<---cut here---end--->8---

Thanks!

Ludo’.

Re: Progress with automating testing of patches

[Christopher Baines]

Christopher Baines  writes:

> Since it's easier to iterate once there's something visible, I've just
> stuck what I've got so far on the internet, it's available at:
>
>   https://qa.guix.gnu.org/
>
> The code is here [3] and I've put a list of ideas in the README.
>
> 3: https://git.cbaines.net/guix/qa-frontpage/about/
>
> Currently, there's a page which lists issues, and pages for individual
> issues that show build and lint warning changes. Behind the scenes, it's
> also submitting builds to the build coordinator for the packages
> affected by patches (for x86_64-linux, i686-linux, aarch64-linux and
> armhf-linux).
>
> As I've been developing it, I've been looking at various recent patch
> series, and it seems like this is already useful. It's reassuring when
> reviewing patches to see that packages still build on the architectures
> being tested. Also, unlike earlier prototype patch testing setups,
> because the builds are now happening on a default substitute server,
> there should be some benefits already with substitutes being available
> at the point the patches are merged.
>
> This is very much still at the prototype stage though, many pages will
> timeout or just fail to load due to an error.
>
> Let me know if you have any comments or questions? If you want to try
> and work on adding any features, I should also have time to try and help
> out as well, so just let me know you're interested.

There's been some more progress now since I sent this email, so I
thought I'd send an update.

I've started to try and show some overall status for each issue, that's
the green circle which can appear on the patches page. I want to try and
make this available as an image as well, so that it can be included on
the issue page on issues.guix.gnu.org, linking up qa.guix.gnu.org and
issues.guix.gnu.org.

I've also managed to use the GraphQL API for issues.guix.gnu.org to
fetch tags, which is very exciting as it makes it possible to do all
kinds of things I think. Starting with showing specific tags on the
qa.guix.gnu.org pages, and ending maybe with replacing Patchwork with
data provided by Mumi.

Also, while builds for patches were happening, there's now support for
submitting builds for a branch. This is the case for staging, so these
builds are now happening. It'll probably take many days, maybe even
weeks for all the builds to happen, but at this point I'm mostly
interested in getting the software working so that qa.guix.gnu.org is
really useful for working with branches going forward.

I also covered this qa.guix.gnu.org thing in the talk I gave yesterday
at the 10 Years of Guix event. See the email with this subject [1] for
some very rough notes from the discussion that happened in the late
afternoon. There's a list of seemingly actionable things in there, some
of which relate to the qa.guix.gnu.org site. In particular, I'll try and
push the repository to Savannah at some point this week, so let me know
if you have thoughts on changing the name (currently qa-frontpage)
before I do.

1: Notes from discussion on Quality Assurance from the 10 Years of Guix event

Do let me know if you have any comments or questions!

Thanks,

Chris


signature.asc
Description: PGP signature