subject:"\[Wikitech\-l\] Re\: Recreating CI with Quibble"

[Wikitech-l] Re: Recreating CI with Quibble

2024-04-22 Thread Arthur Taylor

Hi again,

My bad - `quibble -c bash` does launch a shell with the services running. I
just needed to add `-it` to the docker invocation. That makes things much
simpler - thank you!

Arthur


Am Mo., 22. Apr. 2024 um 09:03 Uhr schrieb Arthur Taylor <
arthur.tay...@wikimedia.de>:

> Hi Antoine,
>
> Thanks for the positive feedback! I'll make a PR to move my documentation
> edits over to the Quibble repo - thanks for the pointer there.
>
> Happy you like the Quabble script - it's definitely not intended as a
> long-term solution for anything - I just wrote it as a quick way to get my
> local setup to do what I wanted. I would be happy to take a look at
> contributing to Quibble itself and making it a bit more friendly for
> debugging purposes.
>
> I tried `quibble -c bash` this morning, or more completely I tried:
>
> ```
> docker run --entrypoint=quibble-with-supervisord --tmpfs
> /workspace/db:size=320M --volume "$(pwd)"/src:/workspace/src
>  --volume "$(pwd)"/cache:/cache  --volume "$(pwd)"/log:/workspace/log
> --volume "$(pwd)"/ref:/srv/git:ro--cap-add=SYS_PTRACE   --security-opt
> seccomp=unconfined   --env-file=env-Wikibase-Master-20240410
>  --init --rm
> docker-registry.wikimedia.org/releng/quibble-buster-php74:1.7.0-s1
>  --packages-source composer  --db mysql --db-dir /workspace/db
>  --git-parallel=8 -c bash
> ```
>
> and didn't get a shell. The command just exited immediately:
>
> ```
> ...
> INFO:quibble.commands:>>> Start: User commands: bash
> INFO:quibble.commands:User commands, working directory: /workspace/src
> INFO:quibble.commands:bash
> INFO:quibble.commands:<<< Finish: User commands: bash, in 0.006 s
> INFO:quibble.commands:Shutting down backends:  http://127.0.0.1:9413 /workspace/src>  
> ...
> ```
>
> so I'm not sure that addresses my wish to have an interactive debugging
> environment with the backends running. But maybe I'm doing something wrong
> there. Pointers welcome.
>
> Will try and get to the documentation this week,
>
> Many thanks,
>
> Arthur
>
>
> Am Fr., 19. Apr. 2024 um 14:50 Uhr schrieb Antoine Musso :
>
>> Le 19/04/2024 à 11:03, Arthur Taylor a écrit :
>> > Hi Sebastian,
>> >
>> > I've also been playing around with debugging CI images on my local
>> > machine recently. I've pushed an update to the Quibble docs on
>> > Mediawiki [1]. Hopefully that's helpful - feedback and changes welcome!
>> >
>> > Thanks,
>> >
>> > Arthur
>> >
>> > [1] https://www.mediawiki.org/wiki/Continuous_integration/Quibble
>>
>> Hello,
>>
>> That documentation was merely an helper for people maintaining the CI
>> stack linking to the releasing doc which is in the repository, how to
>> rebuild the CI and then switch the jobs.
>>
>> The Quibble manual / user documentation is inside the source repository
>> under the doc/source directory
>>
>> https://gerrit.wikimedia.org/g/integration/quibble/+/refs/heads/master/doc/source
>> .  It ends up being published at https://doc.wikimedia.org/quibble/
>> which is the canonical place.
>>
>> The wiki edit at
>>
>> https://www.mediawiki.org/w/index.php?title=Continuous_integration/Quibble=prev=6476384
>> <
>> https://www.mediawiki.org/w/index.php?title=Continuous_integration/Quibble=prev=6476384>
>>
>> is amazing, and it would be really great to have it moved to the source
>> repository.  I like the Quabble.py script to pause before running tests,
>> though one can also ask Quibble to run bash as a test command: `quibble
>> -c bash`.
>>
>> Antoine "hashar" Musso
>>
>>
>
> --
> Arthur Taylor
> Senior Software Engineer
>
> Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin
> Phone: +49 (0)30-577 11 62-0
> https://wikimedia.de
>
> Keep up to date! Current news and exciting stories about Wikimedia,
> Wikipedia and Free Knowledge in our newsletter (in German): Subscribe now
> .
>
> Imagine a world in which every single human being can freely share in the
> sum of all knowledge. Help us to achieve our vision!
> https://spenden.wikimedia.de
>
> Wikimedia Deutschland — Gesellschaft zur Förderung Freien Wissens e. V.
> Eingetragen im Vereinsregister des Amtsgerichts Charlottenburg, VR 23855 B.
> Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin,
> Steuernummer 27/029/42207. Geschäftsführende Vorstände: Franziska Heine,
> Dr. Christian Humborg
>


-- 
Arthur Taylor
Senior Software Engineer

Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Phone: +49 (0)30-577 11 62-0
https://wikimedia.de

Keep up to date! Current news and exciting stories about Wikimedia,
Wikipedia and Free Knowledge in our newsletter (in German): Subscribe now
.

Imagine a world in which every single human being can freely share in the
sum of all knowledge. Help us to achieve our vision!
https://spenden.wikimedia.de

Wikimedia Deutschland — Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts

[Wikitech-l] Re: Recreating CI with Quibble

2024-04-22 Thread Arthur Taylor

Hi Antoine,

Thanks for the positive feedback! I'll make a PR to move my documentation
edits over to the Quibble repo - thanks for the pointer there.

Happy you like the Quabble script - it's definitely not intended as a
long-term solution for anything - I just wrote it as a quick way to get my
local setup to do what I wanted. I would be happy to take a look at
contributing to Quibble itself and making it a bit more friendly for
debugging purposes.

I tried `quibble -c bash` this morning, or more completely I tried:

```
docker run --entrypoint=quibble-with-supervisord --tmpfs
/workspace/db:size=320M --volume "$(pwd)"/src:/workspace/src
 --volume "$(pwd)"/cache:/cache  --volume "$(pwd)"/log:/workspace/log
--volume "$(pwd)"/ref:/srv/git:ro--cap-add=SYS_PTRACE   --security-opt
seccomp=unconfined   --env-file=env-Wikibase-Master-20240410
 --init --rm
docker-registry.wikimedia.org/releng/quibble-buster-php74:1.7.0-s1
 --packages-source composer  --db mysql --db-dir /workspace/db
 --git-parallel=8 -c bash
```

and didn't get a shell. The command just exited immediately:

```
...
INFO:quibble.commands:>>> Start: User commands: bash
INFO:quibble.commands:User commands, working directory: /workspace/src
INFO:quibble.commands:bash
INFO:quibble.commands:<<< Finish: User commands: bash, in 0.006 s
INFO:quibble.commands:Shutting down backends: http://127.0.0.1:9413 /workspace/src>  
...
```

so I'm not sure that addresses my wish to have an interactive debugging
environment with the backends running. But maybe I'm doing something wrong
there. Pointers welcome.

Will try and get to the documentation this week,

Many thanks,

Arthur


Am Fr., 19. Apr. 2024 um 14:50 Uhr schrieb Antoine Musso :

> Le 19/04/2024 à 11:03, Arthur Taylor a écrit :
> > Hi Sebastian,
> >
> > I've also been playing around with debugging CI images on my local
> > machine recently. I've pushed an update to the Quibble docs on
> > Mediawiki [1]. Hopefully that's helpful - feedback and changes welcome!
> >
> > Thanks,
> >
> > Arthur
> >
> > [1] https://www.mediawiki.org/wiki/Continuous_integration/Quibble
>
> Hello,
>
> That documentation was merely an helper for people maintaining the CI
> stack linking to the releasing doc which is in the repository, how to
> rebuild the CI and then switch the jobs.
>
> The Quibble manual / user documentation is inside the source repository
> under the doc/source directory
>
> https://gerrit.wikimedia.org/g/integration/quibble/+/refs/heads/master/doc/source
> .  It ends up being published at https://doc.wikimedia.org/quibble/
> which is the canonical place.
>
> The wiki edit at
>
> https://www.mediawiki.org/w/index.php?title=Continuous_integration/Quibble=prev=6476384
> <
> https://www.mediawiki.org/w/index.php?title=Continuous_integration/Quibble=prev=6476384>
>
> is amazing, and it would be really great to have it moved to the source
> repository.  I like the Quabble.py script to pause before running tests,
> though one can also ask Quibble to run bash as a test command: `quibble
> -c bash`.
>
> Antoine "hashar" Musso
>
>

-- 
Arthur Taylor
Senior Software Engineer

Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Phone: +49 (0)30-577 11 62-0
https://wikimedia.de

Keep up to date! Current news and exciting stories about Wikimedia,
Wikipedia and Free Knowledge in our newsletter (in German): Subscribe now
.

Imagine a world in which every single human being can freely share in the
sum of all knowledge. Help us to achieve our vision!
https://spenden.wikimedia.de

Wikimedia Deutschland — Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Charlottenburg, VR 23855 B.
Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin,
Steuernummer 27/029/42207. Geschäftsführende Vorstände: Franziska Heine,
Dr. Christian Humborg
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: Recreating CI with Quibble

2024-04-19 Thread Antoine Musso


Le 19/04/2024 à 10:42, Sebastian Berlin a écrit :
I'm trying to recreate a CI build to debug things locally. I've 
followed the instructions under 
https://doc.wikimedia.org/quibble/#reproducing-a-ci-build, but the 
command for listing Docker images doesn't work. There are no images 
with "quibble" in the name. Where can I find the available Quibble images?


Hello,

Quibble is not very user friendly. I started it during the 2017 Vienna 
hackathon as a quick hack to unify the complicated stack of repos, 
shell, python, Jenkins job into a single place. That has served to 
reduce the CI complexity and made the jobs quite simpler.


Overall I can't say it is very pleasant to use as a human, there are a 
few oddities such as:


- cleaning the dependencies and reinstalling everytime (use --skip-deps)
- misnamed options such as --skip-zuul to prevent refetching and doing a 
git-clean of everything
- one need to set ZUUL_PROJECT to trigger the logic that runs 
extension/skin tests

And more :)

I have sent a few patches recently to slightly improve it such as:

- automatically deleting the autogenerated LocalSettings.php instead of 
having the MediaWiki installer to bail out after some minutes.


- Adding --change  which retrieve from Gerrit the 
metadata of the given change and initialize the ZUUL_* environment 
variables (though I think one still has to set ZUUL_PROJECT to trigger 
the proper logic ...)


Back on topic, the documentation about reproducing a CI build deserves 
to be refreshed:


- the /v2/_catalog registry endpoint has evolved and is paginated 
nowadays, see 
https://distribution.github.io/distribution/spec/api/#listing-repositories 
or look at which image is currently defined in CI ( 
https://gerrit.wikimedia.org/g/integration/config/+/refs/heads/master/jjb/mediawiki.yaml 
).


Note that CI does a merge commit of the propose patch or patches against 
the tip of the target branch. That is done by processes named 
zuul-merger which exposes the resulting merges using git-daemon. Those 
repositories are not publicly available, though they should be reachable 
from a WMCS instance.


I am more than happy to assist and chat about Quibble. I will very 
warmly welcome any help on that front :)


Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: Recreating CI with Quibble

2024-04-19 Thread Antoine Musso


Le 19/04/2024 à 11:03, Arthur Taylor a écrit :

Hi Sebastian,

I've also been playing around with debugging CI images on my local 
machine recently. I've pushed an update to the Quibble docs on 
Mediawiki [1]. Hopefully that's helpful - feedback and changes welcome!


Thanks,

Arthur

[1] https://www.mediawiki.org/wiki/Continuous_integration/Quibble


Hello,

That documentation was merely an helper for people maintaining the CI 
stack linking to the releasing doc which is in the repository, how to 
rebuild the CI and then switch the jobs.


The Quibble manual / user documentation is inside the source repository 
under the doc/source directory 
https://gerrit.wikimedia.org/g/integration/quibble/+/refs/heads/master/doc/source 
.  It ends up being published at https://doc.wikimedia.org/quibble/ 
which is the canonical place.


The wiki edit at 
https://www.mediawiki.org/w/index.php?title=Continuous_integration/Quibble=prev=6476384 
 
is amazing, and it would be really great to have it moved to the source 
repository.  I like the Quabble.py script to pause before running tests, 
though one can also ask Quibble to run bash as a test command: `quibble 
-c bash`.


Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: Recreating CI with Quibble

2024-04-19 Thread Sebastian Berlin

Thank you both 

I'll have a look at the updated documentation.

*Sebastian Berlin*
Utvecklare/*Developer*
Wikimedia Sverige (WMSE)

E-post/*E-Mail*: sebastian.ber...@wikimedia.se
Telefon/*Phone*: (+46) 0707 - 92 03 84


On Fri, 19 Apr 2024 at 11:03, Arthur Taylor 
wrote:

> Hi Sebastian,
>
> I've also been playing around with debugging CI images on my local machine
> recently. I've pushed an update to the Quibble docs on Mediawiki [1].
> Hopefully that's helpful - feedback and changes welcome!
>
> Thanks,
>
> Arthur
>
> [1] https://www.mediawiki.org/wiki/Continuous_integration/Quibble
>
> Am Fr., 19. Apr. 2024 um 10:56 Uhr schrieb Yiannis Giannelos <
> jgianne...@wikimedia.org>:
>
>> Hi, indeed the API request mentioned in the docs doesn't show up the
>> quibble images.
>>
>> The docker registry mentioned in the docs is this one here [1] so you can
>> search in this page for the available versions of the quibble images.
>> For example this one [2] is built using debian buster and php 8.1.
>>
>> [1] https://docker-registry.wikimedia.org/
>> [2]
>> https://docker-registry.wikimedia.org/releng/quibble-buster-php81/tags/
>>
>> On Fri, Apr 19, 2024 at 10:43 AM Sebastian Berlin <
>> sebastian.ber...@wikimedia.se> wrote:
>>
>>> I'm trying to recreate a CI build to debug things locally. I've followed
>>> the instructions under
>>> https://doc.wikimedia.org/quibble/#reproducing-a-ci-build, but the
>>> command for listing Docker images doesn't work. There are no images with
>>> "quibble" in the name. Where can I find the available Quibble images?
>>>
>>> *Sebastian Berlin*
>>> Utvecklare/*Developer*
>>> Wikimedia Sverige (WMSE)
>>>
>>> E-post/*E-Mail*: sebastian.ber...@wikimedia.se
>>> Telefon/*Phone*: (+46) 0707 - 92 03 84
>>> ___
>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>> To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
>>>
>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>
>> ___
>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>> To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
>>
>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>
>
>
> --
> Arthur Taylor
> Senior Software Engineer
>
> Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin
> Phone: +49 (0)30-577 11 62-0
> https://wikimedia.de
>
> Keep up to date! Current news and exciting stories about Wikimedia,
> Wikipedia and Free Knowledge in our newsletter (in German): Subscribe now
> .
>
> Imagine a world in which every single human being can freely share in the
> sum of all knowledge. Help us to achieve our vision!
> https://spenden.wikimedia.de
>
> Wikimedia Deutschland — Gesellschaft zur Förderung Freien Wissens e. V.
> Eingetragen im Vereinsregister des Amtsgerichts Charlottenburg, VR 23855 B.
> Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin,
> Steuernummer 27/029/42207. Geschäftsführende Vorstände: Franziska Heine,
> Dr. Christian Humborg
> ___
> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
> To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: Recreating CI with Quibble

2024-04-19 Thread Arthur Taylor

Hi Sebastian,

I've also been playing around with debugging CI images on my local machine
recently. I've pushed an update to the Quibble docs on Mediawiki [1].
Hopefully that's helpful - feedback and changes welcome!

Thanks,

Arthur

[1] https://www.mediawiki.org/wiki/Continuous_integration/Quibble

Am Fr., 19. Apr. 2024 um 10:56 Uhr schrieb Yiannis Giannelos <
jgianne...@wikimedia.org>:

> Hi, indeed the API request mentioned in the docs doesn't show up the
> quibble images.
>
> The docker registry mentioned in the docs is this one here [1] so you can
> search in this page for the available versions of the quibble images.
> For example this one [2] is built using debian buster and php 8.1.
>
> [1] https://docker-registry.wikimedia.org/
> [2]
> https://docker-registry.wikimedia.org/releng/quibble-buster-php81/tags/
>
> On Fri, Apr 19, 2024 at 10:43 AM Sebastian Berlin <
> sebastian.ber...@wikimedia.se> wrote:
>
>> I'm trying to recreate a CI build to debug things locally. I've followed
>> the instructions under
>> https://doc.wikimedia.org/quibble/#reproducing-a-ci-build, but the
>> command for listing Docker images doesn't work. There are no images with
>> "quibble" in the name. Where can I find the available Quibble images?
>>
>> *Sebastian Berlin*
>> Utvecklare/*Developer*
>> Wikimedia Sverige (WMSE)
>>
>> E-post/*E-Mail*: sebastian.ber...@wikimedia.se
>> Telefon/*Phone*: (+46) 0707 - 92 03 84
>> ___
>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>> To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
>>
>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>
> ___
> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
> To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

-- 
Arthur Taylor
Senior Software Engineer

Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Phone: +49 (0)30-577 11 62-0
https://wikimedia.de

Keep up to date! Current news and exciting stories about Wikimedia,
Wikipedia and Free Knowledge in our newsletter (in German): Subscribe now
.

Imagine a world in which every single human being can freely share in the
sum of all knowledge. Help us to achieve our vision!
https://spenden.wikimedia.de

Wikimedia Deutschland — Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Charlottenburg, VR 23855 B.
Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin,
Steuernummer 27/029/42207. Geschäftsführende Vorstände: Franziska Heine,
Dr. Christian Humborg
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: Recreating CI with Quibble

2024-04-19 Thread Yiannis Giannelos

Hi, indeed the API request mentioned in the docs doesn't show up the
quibble images.

The docker registry mentioned in the docs is this one here [1] so you can
search in this page for the available versions of the quibble images.
For example this one [2] is built using debian buster and php 8.1.

[1] https://docker-registry.wikimedia.org/
[2] https://docker-registry.wikimedia.org/releng/quibble-buster-php81/tags/

On Fri, Apr 19, 2024 at 10:43 AM Sebastian Berlin <
sebastian.ber...@wikimedia.se> wrote:

> I'm trying to recreate a CI build to debug things locally. I've followed
> the instructions under
> https://doc.wikimedia.org/quibble/#reproducing-a-ci-build, but the
> command for listing Docker images doesn't work. There are no images with
> "quibble" in the name. Where can I find the available Quibble images?
>
> *Sebastian Berlin*
> Utvecklare/*Developer*
> Wikimedia Sverige (WMSE)
>
> E-post/*E-Mail*: sebastian.ber...@wikimedia.se
> Telefon/*Phone*: (+46) 0707 - 92 03 84
> ___
> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
> To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: Recreating CI with Quibble

[Wikitech-l] Re: Recreating CI with Quibble

[Wikitech-l] Re: Recreating CI with Quibble

[Wikitech-l] Re: Recreating CI with Quibble

[Wikitech-l] Re: Recreating CI with Quibble

[Wikitech-l] Re: Recreating CI with Quibble

[Wikitech-l] Re: Recreating CI with Quibble

7 matches

Site Navigation

Mail list logo

Footer information