Re: [RFC qemu.qmp PATCH 00/24] Python: Fork qemu.qmp Python lib into independent repo
On Thu, Dec 16, 2021 at 5:41 AM Daniel P. Berrangé wrote: > > On Wed, Dec 15, 2021 at 04:06:10PM -0500, John Snow wrote: > > (2) To ask for permission to become the maintainer of a > > 'qemu-project/qemu.qmp' repository, where I would like to host this > > subproject. > > I'd say we need 3 designated maintainers as a minimum for redundancy. Let's see how many people we can get to volunteer. If it's part of the qemu-project umbrella, I assume anyone with access to the entire project as a whole can step in and re-assign permissions and do emergency actions as needed, including stsquad, stefanha, etc. That ought to be enough to get moving. If we want to add anyone for operational redundancy, I'd be happy to add as many people that volunteer :) So: any volunteers? Cleber, Beraldo, Wainer? Anyone else? I anticipate this to be extremely low traffic after the churn of splitting it out is done with, it shouldn't be a big daily cost -- but having some familiarity with the code would be good in case you need to accept some urgent patches here and there. --js
Re: [RFC qemu.qmp PATCH 00/24] Python: Fork qemu.qmp Python lib into independent repo
On Thu, Dec 16, 2021 at 5:41 AM Daniel P. Berrangé wrote: > On Wed, Dec 15, 2021 at 04:06:10PM -0500, John Snow wrote: > > Hi, this series is part of an effort to publish the qemu.qmp package on > > PyPI. It is the second of three series to complete this work: > > > > (1) Switch the new Async QMP library in to python/qemu/qmp > > --> (2) Fork python/qemu/qmp out into its own repository, > > with updated GitLab CI/CD targets to build packages. > > (3) Update qemu.git to install qemu.qmp from PyPI, > > and then delete python/qemu/qmp. > > > > This series is not meant to apply to qemu.git, rather -- it's the series > > that performs the split and would apply to a brand new repository. > > > > I am submitting it to the QEMU mailing list for these reasons: > > > > (1) To more broadly announce my intentions, and as reference alongside > > series #1 and #3 detailed above. > > > > (2) To ask for permission to become the maintainer of a > > 'qemu-project/qemu.qmp' repository, where I would like to host this > > subproject. > > I'd say we need 3 designated maintainers as a minimum for redundancy. > Fine by me -- I'd like to nominate Cleber as my current co-maintainer of python/, but that leaves a third spot open. Cleber may decide to nominate someone else who is working on Avocado, too -- that'd be good too. If there was a third person who wasn't @redhat.com, that'd be nice, but nobody comes to mind right away. Any volunteers? Also, I can hand over control of the PyPI project(s) to the conservancy and use revocable auth tokens to perform releases. We'll cross that bridge when we get there, but I've looked into it. > > > (3) To ask for review on the README.rst file which details my intended > > contribution guidelines for this subproject. > > > > (4) To ask for review on the .gitlab-ci.d/ files and other repo-level > > CI/CD ephemera, including and especially the docs-building process. I > > think the generated docs are still ugly, and I'd like to upload them to > > readthedocs, among other things -- hence the RFC quality of this series. > > > Some review/RFC notes: > > > > - I use jsnow/qemu.qmp as the repo name throughout the series; that will > > have to be changed eventually, but for the purposes of prototyping, it > > was nicer to have a fully working series. > > > > - I'm planning on using gitlab issues and MRs for the subproject. > > Great ! > > It's just easier for me, and I suspect it would be easier for non-QEMU contributors to use. I'm starting to try and target people that sit a little further out from our core project, so it seemed like it'd make the most sense. I'll have to work out how to announce changes to the list, though ... maybe I'll have a bot announce merge requests to the mailing list, I'm not sure. > > - I plan to version this lib independently, starting at 0.0.1 for the > > initial public release and bumping only the micro version for every > > last release. I plan to bump the minor version once it hits a "beta" > > state. There will be no cross-versioning against QEMU. I don't plan to > > publish new releases during QEMU freezes. > > IMHO if we're saying that QEMU is going to use this library straight > from PyPI from the start, then we're defacto considering it staable > from the start too. We can't accept changes published to PyPI that > are going to be incompatible with existing QEMU. > > If that isn't acceptable, then QEMU is going to have to be pinned to > a very specific version from PyPi, and explicitly not pull the > latest. > > Right, I was thinking of pinning against a specific version. I want to retain the freedom to change the API for a little while. I was worried that if I tried to make it perfect before publishing it, that I'd never actually make it perfect OR publish it. My plan is something like this: - Increment the micro version for any change during the "alpha" period - Once I remove legacy.py and add a proper sync layer (Which may involve some rework of how the event listeners are factored) I want to version at 0.1.0 and call it "beta". From there, compatible changes and extensions will bump the micro and incompatible changes will bump the minor. - After a QEMU release or two with the beta version and no major problems, I'll probably bump it to 1.0.0, call it stable, and then use semver "as normal" for all future releases. > > - Docs are not yet uploaded anywhere (GitLab pages, readthedocs?) > > Since we're already using gitlab, personally I'd just setup a 'pages' > job and assign a qemu.org sub-domain to gitlab pages service. > > I'll work on a pages job at least, then. I'll look at what you've written for qemu and qemu-web and do some copy-pasting. > > - Tags on a commit trigger two pipelines; this causes one of the package > > builds to fail as the version number will be duplicated in this > > case. Not entirely sure how I want to fix this yet ... > > If you dont have any 'rules:' stanza gitlab creates a pipeline
Re: [RFC qemu.qmp PATCH 00/24] Python: Fork qemu.qmp Python lib into independent repo
On Wed, Dec 15, 2021 at 04:06:10PM -0500, John Snow wrote: > Hi, this series is part of an effort to publish the qemu.qmp package on > PyPI. It is the second of three series to complete this work: > > (1) Switch the new Async QMP library in to python/qemu/qmp > --> (2) Fork python/qemu/qmp out into its own repository, > with updated GitLab CI/CD targets to build packages. > (3) Update qemu.git to install qemu.qmp from PyPI, > and then delete python/qemu/qmp. > > This series is not meant to apply to qemu.git, rather -- it's the series > that performs the split and would apply to a brand new repository. > > I am submitting it to the QEMU mailing list for these reasons: > > (1) To more broadly announce my intentions, and as reference alongside > series #1 and #3 detailed above. > > (2) To ask for permission to become the maintainer of a > 'qemu-project/qemu.qmp' repository, where I would like to host this > subproject. I'd say we need 3 designated maintainers as a minimum for redundancy. > (3) To ask for review on the README.rst file which details my intended > contribution guidelines for this subproject. > > (4) To ask for review on the .gitlab-ci.d/ files and other repo-level > CI/CD ephemera, including and especially the docs-building process. I > think the generated docs are still ugly, and I'd like to upload them to > readthedocs, among other things -- hence the RFC quality of this series. > Some review/RFC notes: > > - I use jsnow/qemu.qmp as the repo name throughout the series; that will > have to be changed eventually, but for the purposes of prototyping, it > was nicer to have a fully working series. > > - I'm planning on using gitlab issues and MRs for the subproject. Great ! > - I plan to version this lib independently, starting at 0.0.1 for the > initial public release and bumping only the micro version for every > last release. I plan to bump the minor version once it hits a "beta" > state. There will be no cross-versioning against QEMU. I don't plan to > publish new releases during QEMU freezes. IMHO if we're saying that QEMU is going to use this library straight from PyPI from the start, then we're defacto considering it staable from the start too. We can't accept changes published to PyPI that are going to be incompatible with existing QEMU. If that isn't acceptable, then QEMU is going to have to be pinned to a very specific version from PyPi, and explicitly not pull the latest. > - Docs are not yet uploaded anywhere (GitLab pages, readthedocs?) Since we're already using gitlab, personally I'd just setup a 'pages' job and assign a qemu.org sub-domain to gitlab pages service. > - Tags on a commit trigger two pipelines; this causes one of the package > builds to fail as the version number will be duplicated in this > case. Not entirely sure how I want to fix this yet ... If you dont have any 'rules:' stanza gitlab creates a pipeline for any 'push' event - this means pushes of branch commits or pushes of tags. To remove the duplicates we need to filter based on certain standard variables - CI_COMMIT_BRANCH or CI_COMMIT_TAG rules: - if '$CI_PIPELINE_SOURCE != "push"' when: never - if '$CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH' when: never - if '$CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH' when: on_success - when: never will cull jobs for pushes of branch commit, leaving pipelines for tag pushes. It can get arbitrarily more complicated depending on what you need to achieve. Since we're going to use merge requests, we should be aiming to *NOT* run pipelines on branch commit pushes for forks. We only want pipelines attached to the merge request. You'll need pipelines on pushes of tags for the post-merge publishing jobs potentially, unless you want todo that on a nightly schedule Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
[RFC qemu.qmp PATCH 00/24] Python: Fork qemu.qmp Python lib into independent repo
Hi, this series is part of an effort to publish the qemu.qmp package on PyPI. It is the second of three series to complete this work: (1) Switch the new Async QMP library in to python/qemu/qmp --> (2) Fork python/qemu/qmp out into its own repository, with updated GitLab CI/CD targets to build packages. (3) Update qemu.git to install qemu.qmp from PyPI, and then delete python/qemu/qmp. This series is not meant to apply to qemu.git, rather -- it's the series that performs the split and would apply to a brand new repository. I am submitting it to the QEMU mailing list for these reasons: (1) To more broadly announce my intentions, and as reference alongside series #1 and #3 detailed above. (2) To ask for permission to become the maintainer of a 'qemu-project/qemu.qmp' repository, where I would like to host this subproject. (3) To ask for review on the README.rst file which details my intended contribution guidelines for this subproject. (4) To ask for review on the .gitlab-ci.d/ files and other repo-level CI/CD ephemera, including and especially the docs-building process. I think the generated docs are still ugly, and I'd like to upload them to readthedocs, among other things -- hence the RFC quality of this series. Some review/RFC notes: - I use jsnow/qemu.qmp as the repo name throughout the series; that will have to be changed eventually, but for the purposes of prototyping, it was nicer to have a fully working series. - I'm planning on using gitlab issues and MRs for the subproject. - I plan to version this lib independently, starting at 0.0.1 for the initial public release and bumping only the micro version for every last release. I plan to bump the minor version once it hits a "beta" state. There will be no cross-versioning against QEMU. I don't plan to publish new releases during QEMU freezes. - Check out a completed pipeline here: https://gitlab.com/jsnow/qemu.qmp/-/pipelines/430528258 It offers build artifacts, junit xml artifacts and GitLab test-level-view into the avocado unit tests. The build container is uploaded to GitLab's container registry and can be used to reproduce potential build/packaging errors. Every pipeline will produce built python packages and upload them to the GitLab package repository, see https://gitlab.com/jsnow/qemu.qmp/-/packages Known problems: - Sphinx output is still subjectively ugly, with too many layers of nesting - Docs are not yet uploaded anywhere (GitLab pages, readthedocs?) - Tags on a commit trigger two pipelines; this causes one of the package builds to fail as the version number will be duplicated in this case. Not entirely sure how I want to fix this yet ... ~ Happy Holidays ~, --js. John Snow (24): Fork qemu.qmp from qemu.git Update VERSION to 0.0.0a1 Update maintainer metadata Update project description Update project URLs Move README.rst to INDEX.rst and update Move PACKAGE.rst to README.rst and update Update Pipfile.lock Remove sub-dependency pins from Pipfile Add build and test container to gitlab CI configuration Add package build step to GitLab CI GitLab CI: Add check-dco script GitLab CI: Add pipenv and tox tests GitLab CI: Add avocado junit XML output to tests GitLab CI: Publish python packages to GitLab package repo Add setuptools_scm package versioning Makefile: add build and publish targets add Sphinx documentation config stub python: configure sphinx python: adjust apidoc stubs Fix doc cross-reference regressions docs: add Makefile target docs: add doc build to GitLab CI build step v0.0.1 .gitignore | 2 +- .gitlab-ci.d/build.yml | 14 ++ .gitlab-ci.d/check-dco.py | 98 ++ .gitlab-ci.d/containers.yml| 28 +++ .gitlab-ci.d/index.yml | 14 ++ .gitlab-ci.d/publish.yml | 11 ++ .gitlab-ci.d/python.Dockerfile | 35 .gitlab-ci.d/test.yml | 74 .gitlab-ci.yml | 3 + INDEX.rst | 64 +++ MANIFEST.in| 4 +- Makefile | 54 +- PACKAGE.rst| 43 - Pipfile| 4 +- Pipfile.lock | 314 ++--- README.rst | 219 ++- VERSION| 1 - avocado.cfg| 7 + docs/Makefile | 20 +++ docs/conf.py | 107 +++ docs/index.rst | 21 +++ docs/make.bat | 35 docs/qemu.qmp.error.rst| 8 + docs/qemu.qmp.events.rst | 7 + docs/qemu.qmp.legacy.rst | 7 + docs/qemu.qmp.message.rst | 8 + docs/qemu.qmp.models.rst | 8 + docs/qemu.qmp.protocol.rst | 9 + docs/qemu.qmp.qmp_client.rst | 8 + docs/qemu.qmp.rst | 24 +++ docs/qemu.qmp.util.rst | 8 +
