Re: Package testing with Salsa CI for new packages

2023-08-21 Thread Paul Boddie
On Monday, 21 August 2023 15:01:26 CEST Carles Pina i Estany wrote:
> 
> If you want, you can simplify more (it's not exactly the same, so it
> might or might not help). There is a way on GitLab to point to the
> latest build of a job. For example, you have the following URL for one
> of the git repos:
>
> https://salsa.debian.org/moin-team/emeraldtree/-/jobs/4575438/artifacts/raw/
aptly
> 
> You could use instead (to avoid the pipeline number):
> https://salsa.debian.org/moin-team/emeraldtree/-/jobs/artifacts/debian/
master/raw/aptly?job=aptly

That is a good idea, since I always intend the latest job to be successful. 
:-)

> Which is a redirect to the latest pipeline. Currently:
> 
> $ curl -s -I
> "https://salsa.debian.org/moin-team/emeraldtree/-/jobs/artifacts/debian/mas
> ter/raw/aptly?job=aptly" | grep -E -i "^(http|location)" HTTP/2 302
> location:
> https://salsa.debian.org/moin-team/emeraldtree/-/jobs/4575438/artifacts/raw/
aptly
> 
> 
> Follows this format:
> BRANCH=debian/master
> DIRECTORY=aptly
> JOB_NAME=aptly
> https://salsa.debian.org/moin-team/emeraldtree/-/jobs/artifacts/${BRANCH}/ra
> w/${DIRECTORY}?job=${JOB_NAME}

Thanks for demonstrating this. I imagine that it is documented somewhere in 
the GitLab manuals, too. One note of caution, though. It seems that using 
these redirected URLs is not necessarily possible with apt, and so it might be 
advisible to obtain the redirected URL as shown in the output above.

In my script I now get curl (and not wget any more) to obtain the final URL to 
provide to the apt configuration, and I also use this job-specific URL to 
obtain each GPG key. I suppose I could change the apt configuration to allow 
redirects instead.

> Just a side note: be careful about expiring artifacts. In some projects
> (settings dependant) only the latest artifact is kept and older ones
> might be expired (deleted) after some time. I don't think that this is
> the case of the moin-team/emeraldtree after a quick check... but I'm
> unsure where this is properly checked on GitLab.

"CI/CD Settings", "Artifacts", "Keep artifacts from most recent successful 
jobs" is set, so I imagine that as long as the latest job is successful, 
everything will still work. Indeed, it is desirable to rely on the latest job 
under such conditions: one would not want a newer dependency build to break 
the Moin package because an older set of artefacts has been deleted.

Thanks once again for the feedback and advice!

Paul




Re: Package testing with Salsa CI for new packages

2023-08-21 Thread Carles Pina i Estany


Hi,

On 20 Aug 2023 at 23:16:57, Paul Boddie wrote:
> On Sunday, 20 August 2023 14:06:37 CEST Carles Pina i Estany wrote:

[...]

Thanks for sharing your path here, and I'm happy that you are reaching
your destination :-)

> autopkgtest:
>   extends: .test-autopkgtest
>   variables:
> SALSA_CI_AUTOPKGTEST_ARGS: '--setup-commands=debian/salsa/add-
> repositories.sh'
> 
> piuparts:
>   extends: .test-piuparts
>   variables:
> SALSA_CI_PIUPARTS_PRE_INSTALL_SCRIPT: 'debian/salsa/add-repositories.sh'
> 
> You can see that by defining the variables to customise the tools, I am able 
> to work with the existing job definitions. This simplifies the CI description 
> file considerably:
> 
> https://salsa.debian.org/moin-team/moin/-/blob/debian/master/debian/salsa-ci.yml
> 
> The script is pretty straightforward, too:
> 
> https://salsa.debian.org/moin-team/moin/-/blob/debian/master/debian/salsa/add-repositories.sh

If you want, you can simplify more (it's not exactly the same, so it
might or might not help). There is a way on GitLab to point to the
latest build of a job. For example, you have the following URL for one
of the git repos:
https://salsa.debian.org/moin-team/emeraldtree/-/jobs/4575438/artifacts/raw/aptly

You could use instead (to avoid the pipeline number):
https://salsa.debian.org/moin-team/emeraldtree/-/jobs/artifacts/debian/master/raw/aptly?job=aptly

Which is a redirect to the latest pipeline. Currently:

$ curl -s -I 
"https://salsa.debian.org/moin-team/emeraldtree/-/jobs/artifacts/debian/master/raw/aptly?job=aptly;
 | grep -E -i "^(http|location)"
HTTP/2 302
location: 
https://salsa.debian.org/moin-team/emeraldtree/-/jobs/4575438/artifacts/raw/aptly


Follows this format:
BRANCH=debian/master
DIRECTORY=aptly
JOB_NAME=aptly
https://salsa.debian.org/moin-team/emeraldtree/-/jobs/artifacts/${BRANCH}/raw/${DIRECTORY}?job=${JOB_NAME}

Just a side note: be careful about expiring artifacts. In some projects
(settings dependant) only the latest artifact is kept and older ones
might be expired (deleted) after some time. I don't think that this is
the case of the moin-team/emeraldtree after a quick check... but I'm
unsure where this is properly checked on GitLab.

[...]

Cheers,

-- 
Carles Pina i Estany
https://carles.pina.cat || Wiktionary translations: https://kamus.pina.cat