subject:"Re\: \[PATCH v2 07\/11\] acpi\/tests\/bits\: add python test that exercizes QEMU bios tables using biosbits"

On Wed, Sep 28, 2022 at 11:39:36AM +0200, Thomas Huth wrote:
> On 28/09/2022 11.35, Michael S. Tsirkin wrote:
> > On Wed, Sep 28, 2022 at 10:31:39AM +0200, Thomas Huth wrote:
> > > On 27/09/2022 23.21, Michael S. Tsirkin wrote:
> > > > On Tue, Sep 27, 2022 at 04:45:09PM +0100, Daniel P. Berrangé wrote:
> > > > > On Tue, Sep 27, 2022 at 07:35:13PM +0530, Ani Sinha wrote:
> > > ...
> > > > > > Alright, .gitlab-ci.yml is produced and the pipeline succeeds.
> > > > > > However, the question still remains, where do we keep the generated
> > > > > > artifacts?
> > > > > 
> > > > > The following link will always reflect the published artifacts from
> > > > > the most recently fully successful CI pipeline, on the 'qemu-bits'
> > > > > branch, and 'qemu-bits-build' CI job:
> > > > > 
> > > > > https://gitlab.com/qemu-project/biosbits-bits/-/jobs/artifacts/qemu-bits/download?job=qemu-bits-build
> > > > > 
> > > > > Tweak as needed if you push the CI to master branch instead. This
> > > > > link can be considered the permanent home of the artifact. I'd just
> > > > > suggest that the QEMU job automatically skip if it fails to download
> > > > > the artifact, as occassionally transient infra errors can impact
> > > > > it.
> > > > 
> > > > This just means once we change the test old qemu source can no longer 
> > > > use it.
> > > > Why is this a good idea? Are we so short on disk space? I thought CPU
> > > > is the limiting factor?
> > > 
> > > FYI, we'll soon be short on disk space, gitlab plans to introduce storage
> > > limits:
> > > 
> > >   https://about.gitlab.com/pricing/faq-paid-storage-transfer/
> > > 
> > >   Thomas
> > 
> > A good reason not to use CI artifacts to store images maybe?
> > I was proposing storing binaries on qemu.org not on gitlab.
> 
> For qemu.org, you should maybe talk to Paolo and Stefan first, I'm not sure
> whether we could allow additional network traffic
> beside the normal release tarballs there...
> 
>  Thomas

I guess we need to design this sensibly to checksum local files
and only fetch if there's change, and that only for
people who work on ACPI.

-- 
MST

Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits

On Wed, Sep 28, 2022 at 10:31:39AM +0200, Thomas Huth wrote:
> On 27/09/2022 23.21, Michael S. Tsirkin wrote:
> > On Tue, Sep 27, 2022 at 04:45:09PM +0100, Daniel P. Berrangé wrote:
> > > On Tue, Sep 27, 2022 at 07:35:13PM +0530, Ani Sinha wrote:
> ...
> > > > Alright, .gitlab-ci.yml is produced and the pipeline succeeds.
> > > > However, the question still remains, where do we keep the generated
> > > > artifacts?
> > > 
> > > The following link will always reflect the published artifacts from
> > > the most recently fully successful CI pipeline, on the 'qemu-bits'
> > > branch, and 'qemu-bits-build' CI job:
> > > 
> > > https://gitlab.com/qemu-project/biosbits-bits/-/jobs/artifacts/qemu-bits/download?job=qemu-bits-build
> > > 
> > > Tweak as needed if you push the CI to master branch instead. This
> > > link can be considered the permanent home of the artifact. I'd just
> > > suggest that the QEMU job automatically skip if it fails to download
> > > the artifact, as occassionally transient infra errors can impact
> > > it.
> > 
> > This just means once we change the test old qemu source can no longer use 
> > it.
> > Why is this a good idea? Are we so short on disk space? I thought CPU
> > is the limiting factor?
> 
> FYI, we'll soon be short on disk space, gitlab plans to introduce storage
> limits:
> 
>  https://about.gitlab.com/pricing/faq-paid-storage-transfer/
> 
>  Thomas

A good reason not to use CI artifacts to store images maybe?
I was proposing storing binaries on qemu.org not on gitlab.

-- 
MST

Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits

2022-09-28 Thread Thomas Huth


On 27/09/2022 23.21, Michael S. Tsirkin wrote:

On Tue, Sep 27, 2022 at 04:45:09PM +0100, Daniel P. Berrangé wrote:

On Tue, Sep 27, 2022 at 07:35:13PM +0530, Ani Sinha wrote:

...

Alright, .gitlab-ci.yml is produced and the pipeline succeeds.
However, the question still remains, where do we keep the generated
artifacts?


The following link will always reflect the published artifacts from
the most recently fully successful CI pipeline, on the 'qemu-bits'
branch, and 'qemu-bits-build' CI job:

https://gitlab.com/qemu-project/biosbits-bits/-/jobs/artifacts/qemu-bits/download?job=qemu-bits-build

Tweak as needed if you push the CI to master branch instead. This
link can be considered the permanent home of the artifact. I'd just
suggest that the QEMU job automatically skip if it fails to download
the artifact, as occassionally transient infra errors can impact
it.


This just means once we change the test old qemu source can no longer use it.
Why is this a good idea? Are we so short on disk space? I thought CPU
is the limiting factor?


FYI, we'll soon be short on disk space, gitlab plans to introduce storage 
limits:


 https://about.gitlab.com/pricing/faq-paid-storage-transfer/

 Thomas

Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits

On Wed, Sep 28, 2022 at 10:31:39AM +0200, Thomas Huth wrote:
> On 27/09/2022 23.21, Michael S. Tsirkin wrote:
> > On Tue, Sep 27, 2022 at 04:45:09PM +0100, Daniel P. Berrangé wrote:
> > > On Tue, Sep 27, 2022 at 07:35:13PM +0530, Ani Sinha wrote:
> ...
> > > > Alright, .gitlab-ci.yml is produced and the pipeline succeeds.
> > > > However, the question still remains, where do we keep the generated
> > > > artifacts?
> > > 
> > > The following link will always reflect the published artifacts from
> > > the most recently fully successful CI pipeline, on the 'qemu-bits'
> > > branch, and 'qemu-bits-build' CI job:
> > > 
> > > https://gitlab.com/qemu-project/biosbits-bits/-/jobs/artifacts/qemu-bits/download?job=qemu-bits-build
> > > 
> > > Tweak as needed if you push the CI to master branch instead. This
> > > link can be considered the permanent home of the artifact. I'd just
> > > suggest that the QEMU job automatically skip if it fails to download
> > > the artifact, as occassionally transient infra errors can impact
> > > it.
> > 
> > This just means once we change the test old qemu source can no longer use 
> > it.
> > Why is this a good idea? Are we so short on disk space? I thought CPU
> > is the limiting factor?
> 
> FYI, we'll soon be short on disk space, gitlab plans to introduce storage
> limits:
> 
>  https://about.gitlab.com/pricing/faq-paid-storage-transfer/

That's the key reason I prefer the binary as CI artifact rather than
in Git. Once checked into git, you can never reclaim that storage
usage, as the git repo is append only, only option is to delete the
repo and recreate.  With CI artifacts we can control exactly which
binaries consume storage quota at any time.

With regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits

On Wed, Sep 28, 2022 at 09:35:59AM +0100, Daniel P. Berrangé wrote:
> On Wed, Sep 28, 2022 at 10:31:39AM +0200, Thomas Huth wrote:
> > On 27/09/2022 23.21, Michael S. Tsirkin wrote:
> > > On Tue, Sep 27, 2022 at 04:45:09PM +0100, Daniel P. Berrangé wrote:
> > > > On Tue, Sep 27, 2022 at 07:35:13PM +0530, Ani Sinha wrote:
> > ...
> > > > > Alright, .gitlab-ci.yml is produced and the pipeline succeeds.
> > > > > However, the question still remains, where do we keep the generated
> > > > > artifacts?
> > > > 
> > > > The following link will always reflect the published artifacts from
> > > > the most recently fully successful CI pipeline, on the 'qemu-bits'
> > > > branch, and 'qemu-bits-build' CI job:
> > > > 
> > > > https://gitlab.com/qemu-project/biosbits-bits/-/jobs/artifacts/qemu-bits/download?job=qemu-bits-build
> > > > 
> > > > Tweak as needed if you push the CI to master branch instead. This
> > > > link can be considered the permanent home of the artifact. I'd just
> > > > suggest that the QEMU job automatically skip if it fails to download
> > > > the artifact, as occassionally transient infra errors can impact
> > > > it.
> > > 
> > > This just means once we change the test old qemu source can no longer use 
> > > it.
> > > Why is this a good idea? Are we so short on disk space? I thought CPU
> > > is the limiting factor?
> > 
> > FYI, we'll soon be short on disk space, gitlab plans to introduce storage
> > limits:
> > 
> >  https://about.gitlab.com/pricing/faq-paid-storage-transfer/
> 
> That's the key reason I prefer the binary as CI artifact rather than
> in Git. Once checked into git, you can never reclaim that storage
> usage, as the git repo is append only, only option is to delete the
> repo and recreate.  With CI artifacts we can control exactly which
> binaries consume storage quota at any time.
> 
> With regards,
> Daniel

I agree binaries in git are a bit of a hack.
But I also feel managing files as part of a test tool is a hack too,
it's an SCM issue.  How about e.g. git-lfs? Seems to be reasonably well
supported on gitlab. There's also gitlab but that seems to be older.


> -- 
> |: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org -o-https://fstop138.berrange.com :|
> |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits

2022-09-28 Thread Thomas Huth

On 28/09/2022 11.35, Michael S. Tsirkin wrote:

On Wed, Sep 28, 2022 at 10:31:39AM +0200, Thomas Huth wrote:

On 27/09/2022 23.21, Michael S. Tsirkin wrote:

On Tue, Sep 27, 2022 at 04:45:09PM +0100, Daniel P. Berrangé wrote:

On Tue, Sep 27, 2022 at 07:35:13PM +0530, Ani Sinha wrote:

...

Alright, .gitlab-ci.yml is produced and the pipeline succeeds.
However, the question still remains, where do we keep the generated
artifacts?

The following link will always reflect the published artifacts from
the most recently fully successful CI pipeline, on the 'qemu-bits'
branch, and 'qemu-bits-build' CI job:

https://gitlab.com/qemu-project/biosbits-bits/-/jobs/artifacts/qemu-bits/download?job=qemu-bits-build

Tweak as needed if you push the CI to master branch instead. This
link can be considered the permanent home of the artifact. I'd just
suggest that the QEMU job automatically skip if it fails to download
the artifact, as occassionally transient infra errors can impact
it.

This just means once we change the test old qemu source can no longer use it.
Why is this a good idea? Are we so short on disk space? I thought CPU
is the limiting factor?

FYI, we'll soon be short on disk space, gitlab plans to introduce storage
limits:

https://about.gitlab.com/pricing/faq-paid-storage-transfer/

Thomas

A good reason not to use CI artifacts to store images maybe?
I was proposing storing binaries on qemu.org not on gitlab.

For qemu.org, you should maybe talk to Paolo and Stefan first, I'm not sure
whether we could allow additional network traffic

beside the normal release tarballs there...

Thomas

Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits

On Wed, Sep 28, 2022 at 07:58:46AM +0100, Daniel P. Berrangé wrote:
> biosbits isn't tied to QEMU versions, it is an entirely separate 3rd
> party project. This binary is just providing the test env, and IIUC,
> control over what executes in this env is still done by the QEMU side
> test scripts. I'm not seeing a coupling here that requires precise
> matching. In any case biosbit is a dead project so does not look
> likely to have any changes.
> 
> If we did want to have different versions though, we can stil
> publish artifacts from different branches of biosbits code. Gitlab
> will preserve & publish the latest artifacts from each branch in
> parallel.
> 
> With regards,
> Daniel

The issue is bugs, testing, support.  If biosbits needs to support old
qemu versions that's a ton of work for no real benefit.  CI is CI, it's
not a binary distribution mechanism, abusing it will just lead to
problems down the road... my $.02

-- 
MST

Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits

On Wed, Sep 28, 2022 at 12:45:46PM +0530, Ani Sinha wrote:
> On Wed, Sep 28, 2022 at 12:28 PM Daniel P. Berrangé  
> wrote:
> >
> > On Tue, Sep 27, 2022 at 05:18:10PM -0400, Michael S. Tsirkin wrote:
> > > On Tue, Sep 27, 2022 at 09:33:27AM +0100, Daniel P. Berrangé wrote:
> > > > On Tue, Sep 27, 2022 at 01:43:15PM +0530, Ani Sinha wrote:
> > > > > On Sun, Sep 18, 2022 at 1:58 AM Michael S. Tsirkin  
> > > > > wrote:
> > > > > >
> > > > > > On Fri, Sep 16, 2022 at 09:30:42PM +0530, Ani Sinha wrote:
> > > > > > > On Thu, Jul 28, 2022 at 12:08 AM Ani Sinha  
> > > > > > > wrote:
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Mon, 25 Jul 2022, Ani Sinha wrote:
> > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Sat, 16 Jul 2022, Michael S. Tsirkin wrote:
> > > > > > > > >
> > > > > > > > > > On Sat, Jul 16, 2022 at 12:06:00PM +0530, Ani Sinha wrote:
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On Fri, Jul 15, 2022 at 11:20 Michael S. Tsirkin 
> > > > > > > > > > >  wrote:
> > > > > > > > > > >
> > > > > > > > > > > On Fri, Jul 15, 2022 at 09:47:27AM +0530, Ani Sinha 
> > > > > > > > > > > wrote:
> > > > > > > > > > > > > Instead of all this mess, can't we just spawn 
> > > > > > > > > > > e.g. "git clone --depth
> > > > > > > > > > > 1"?
> > > > > > > > > > > > > And if the directory exists I would fetch and 
> > > > > > > > > > > checkout.
> > > > > > > > > > > >
> > > > > > > > > > > > There are two reasons I can think of why I do not 
> > > > > > > > > > > like this idea:
> > > > > > > > > > > >
> > > > > > > > > > > > (a) a git clone of a whole directory would download 
> > > > > > > > > > > all versions of the
> > > > > > > > > > > > binary whereas we want only a specific version.
> > > > > > > > > > >
> > > > > > > > > > > You mention shallow clone yourself, and I used 
> > > > > > > > > > > --depth 1 above.
> > > > > > > > > > >
> > > > > > > > > > > > Downloading a single file
> > > > > > > > > > > > by shallow cloning or creating a git archive is 
> > > > > > > > > > > overkill IMHO when a wget
> > > > > > > > > > > > style retrieval works just fine.
> > > > > > > > > > >
> > > > > > > > > > > However, it does not provide for versioning, tagging 
> > > > > > > > > > > etc so you have
> > > > > > > > > > > to implement your own schema.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Hmm I’m not sure if we need all that. Bits has its own 
> > > > > > > > > > > versioning mechanism and
> > > > > > > > > > > I think all we need to do is maintain the same versioning 
> > > > > > > > > > > logic and maintain
> > > > > > > > > > > binaries of different  versions. Do we really need the 
> > > > > > > > > > > power of git/version
> > > > > > > > > > > control here? Dunno.
> > > > > > > > > >
> > > > > > > > > > Well we need some schema. Given we are not using official 
> > > > > > > > > > bits releases
> > > > > > > > > > I don't think we can reuse theirs.
> > > > > > > > >
> > > > > > > > > OK fine. Lets figuire out how to push bits somewhere in 
> > > > > > > > > git.qemu.org and
> > > > > > > > > the binaries in some other repo first. Everything else hinges 
> > > > > > > > > on that. We
> > > > > > > > > can fix the rest of the bits later incrementally.
> > > > > > > >
> > > > > > > > DanPB, any thoughts on putting bits on git.qemu.org or where 
> > > > > > > > and how to
> > > > > > > > keep the binaries?
> > > > > > >
> > > > > > > Can we please conclude on this?
> > > > > > > Peter, can you please fork the repo? I have tried many times to 
> > > > > > > reach
> > > > > > > you on IRC but failed.
> > > > > >
> > > > > > Probably because of travel around KVM forum.
> > > > > >
> > > > > > I think given our CI is under pressure again due to gitlab free tier
> > > > > > limits, tying binaries to CI isn't a great idea at this stage.
> > > > > > Can Ani just upload binaies to qemu.org for now?
> > > > >
> > > > > I agree with Michael here. Having a full ci/cd job for this is
> > > > > overkill IMHO. We should create a repo just for the binaries, have a
> > > > > README there to explain how we generate them and check in new versions
> > > > > as and when needed (it won't be frequent).
> > > > > How about biosbits-bin repo?
> > > >
> > > > If QEMU is hosting binaries, where any part contains GPL code, then we
> > > > need to be providing the full and corresponding source and the build
> > > > scripts needed to re-create the binary. Once we have such scripts it
> > > > should be trivial to trigger that from a CI job. If it isn't then
> > > > we're doing something wrong.  The CI quota is not an issue, because
> > > > this is not a job that we need to run continuously. It can be triggered
> > > > manually as & when we decide we need to refresh the binary, so would
> > > > be a small one-off CI quota hit.
> > > >
> > > > Also note that gitlab is intending to start enforcing

Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits

2022-09-28 Thread Thomas Huth


On 28/09/2022 09.06, Daniel P. Berrangé wrote:

On Tue, Sep 27, 2022 at 06:09:22PM -0400, Michael S. Tsirkin wrote:

On Tue, Sep 27, 2022 at 11:44:56PM +0200, Paolo Bonzini wrote:

I also second the idea of using avocado instead of pytest, by the way.


snip


Problem is I don't think avocado is yet at the level where I can
ask random developers to use it to check their ACPI patches.

I just went ahead and rechecked and the situation isn't much better
yet. I think the focus of avocado is system testing of full guests with
KVM, not unit testing of ACPI.

Let's start with installation on a clean box:


...snip...

Do not do any of this stuff, it is irrelevant to QEMU's needs.
A developer using Avocado with QEMU does nothing more than:

 make check-avocado


Right. And if you want to run individual tests, you can also do it like this:

make check-venv   # Only for the first time
./tests/venv/bin/avocado run tests/avocado/boot_linux.py

Or run tests via tags (very convenient for maintainers):

   ./tests/venv/bin/avocado run -t arch:s390x tests/avocado/

 HTH,
  Thomas

Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits

On Tue, Sep 27, 2022 at 06:09:22PM -0400, Michael S. Tsirkin wrote:
> On Tue, Sep 27, 2022 at 11:44:56PM +0200, Paolo Bonzini wrote:
> > I also second the idea of using avocado instead of pytest, by the way.

snip

> Problem is I don't think avocado is yet at the level where I can
> ask random developers to use it to check their ACPI patches.
> 
> I just went ahead and rechecked and the situation isn't much better
> yet. I think the focus of avocado is system testing of full guests with
> KVM, not unit testing of ACPI.
> 
> Let's start with installation on a clean box:

...snip...

Do not do any of this stuff, it is irrelevant to QEMU's needs.
A developer using Avocado with QEMU does nothing more than:

make check-avocado

The avocado framework itself is setup in a virtual env on first
run by the makefile. Thus the developers don't need to install
nor interact with avocado at all, merely write a test script
and put it in tests/avocado/ in QEMU git.

With regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits

On Tue, Sep 27, 2022 at 05:18:10PM -0400, Michael S. Tsirkin wrote:
> On Tue, Sep 27, 2022 at 09:33:27AM +0100, Daniel P. Berrangé wrote:
> > On Tue, Sep 27, 2022 at 01:43:15PM +0530, Ani Sinha wrote:
> > > On Sun, Sep 18, 2022 at 1:58 AM Michael S. Tsirkin  
> > > wrote:
> > > >
> > > > On Fri, Sep 16, 2022 at 09:30:42PM +0530, Ani Sinha wrote:
> > > > > On Thu, Jul 28, 2022 at 12:08 AM Ani Sinha  wrote:
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Mon, 25 Jul 2022, Ani Sinha wrote:
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Sat, 16 Jul 2022, Michael S. Tsirkin wrote:
> > > > > > >
> > > > > > > > On Sat, Jul 16, 2022 at 12:06:00PM +0530, Ani Sinha wrote:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Fri, Jul 15, 2022 at 11:20 Michael S. Tsirkin 
> > > > > > > > >  wrote:
> > > > > > > > >
> > > > > > > > > On Fri, Jul 15, 2022 at 09:47:27AM +0530, Ani Sinha wrote:
> > > > > > > > > > > Instead of all this mess, can't we just spawn e.g. 
> > > > > > > > > "git clone --depth
> > > > > > > > > 1"?
> > > > > > > > > > > And if the directory exists I would fetch and 
> > > > > > > > > checkout.
> > > > > > > > > >
> > > > > > > > > > There are two reasons I can think of why I do not like 
> > > > > > > > > this idea:
> > > > > > > > > >
> > > > > > > > > > (a) a git clone of a whole directory would download all 
> > > > > > > > > versions of the
> > > > > > > > > > binary whereas we want only a specific version.
> > > > > > > > >
> > > > > > > > > You mention shallow clone yourself, and I used --depth 1 
> > > > > > > > > above.
> > > > > > > > >
> > > > > > > > > > Downloading a single file
> > > > > > > > > > by shallow cloning or creating a git archive is 
> > > > > > > > > overkill IMHO when a wget
> > > > > > > > > > style retrieval works just fine.
> > > > > > > > >
> > > > > > > > > However, it does not provide for versioning, tagging etc 
> > > > > > > > > so you have
> > > > > > > > > to implement your own schema.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Hmm I’m not sure if we need all that. Bits has its own 
> > > > > > > > > versioning mechanism and
> > > > > > > > > I think all we need to do is maintain the same versioning 
> > > > > > > > > logic and maintain
> > > > > > > > > binaries of different  versions. Do we really need the power 
> > > > > > > > > of git/version
> > > > > > > > > control here? Dunno.
> > > > > > > >
> > > > > > > > Well we need some schema. Given we are not using official bits 
> > > > > > > > releases
> > > > > > > > I don't think we can reuse theirs.
> > > > > > >
> > > > > > > OK fine. Lets figuire out how to push bits somewhere in 
> > > > > > > git.qemu.org and
> > > > > > > the binaries in some other repo first. Everything else hinges on 
> > > > > > > that. We
> > > > > > > can fix the rest of the bits later incrementally.
> > > > > >
> > > > > > DanPB, any thoughts on putting bits on git.qemu.org or where and 
> > > > > > how to
> > > > > > keep the binaries?
> > > > >
> > > > > Can we please conclude on this?
> > > > > Peter, can you please fork the repo? I have tried many times to reach
> > > > > you on IRC but failed.
> > > >
> > > > Probably because of travel around KVM forum.
> > > >
> > > > I think given our CI is under pressure again due to gitlab free tier
> > > > limits, tying binaries to CI isn't a great idea at this stage.
> > > > Can Ani just upload binaies to qemu.org for now?
> > > 
> > > I agree with Michael here. Having a full ci/cd job for this is
> > > overkill IMHO. We should create a repo just for the binaries, have a
> > > README there to explain how we generate them and check in new versions
> > > as and when needed (it won't be frequent).
> > > How about biosbits-bin repo?
> > 
> > If QEMU is hosting binaries, where any part contains GPL code, then we
> > need to be providing the full and corresponding source and the build
> > scripts needed to re-create the binary. Once we have such scripts it
> > should be trivial to trigger that from a CI job. If it isn't then
> > we're doing something wrong.  The CI quota is not an issue, because
> > this is not a job that we need to run continuously. It can be triggered
> > manually as & when we decide we need to refresh the binary, so would
> > be a small one-off CI quota hit.
> > 
> > Also note that gitlab is intending to start enforcing storage quota
> > on projects in the not too distant future. This makes it unappealing
> > to store binaries in git repos, unless we genuinely need the ability
> > to access historical versions of the binary. I don't believe we need
> > that for biosbits.
> > 
> > The binary can be published as a CI artifact and accessed directly
> > from the latest artifact download link. This ensures we only consume
> > quota for the most recently published binary artifact. So I don't see
> > a compelling reason to upload binaries into git.
>
> I don't really care where we upload

Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits

2022-09-28 Thread Ani Sinha

On Wed, Sep 28, 2022 at 12:28 PM Daniel P. Berrangé  wrote:
>
> On Tue, Sep 27, 2022 at 05:18:10PM -0400, Michael S. Tsirkin wrote:
> > On Tue, Sep 27, 2022 at 09:33:27AM +0100, Daniel P. Berrangé wrote:
> > > On Tue, Sep 27, 2022 at 01:43:15PM +0530, Ani Sinha wrote:
> > > > On Sun, Sep 18, 2022 at 1:58 AM Michael S. Tsirkin  
> > > > wrote:
> > > > >
> > > > > On Fri, Sep 16, 2022 at 09:30:42PM +0530, Ani Sinha wrote:
> > > > > > On Thu, Jul 28, 2022 at 12:08 AM Ani Sinha  wrote:
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Mon, 25 Jul 2022, Ani Sinha wrote:
> > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Sat, 16 Jul 2022, Michael S. Tsirkin wrote:
> > > > > > > >
> > > > > > > > > On Sat, Jul 16, 2022 at 12:06:00PM +0530, Ani Sinha wrote:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Fri, Jul 15, 2022 at 11:20 Michael S. Tsirkin 
> > > > > > > > > >  wrote:
> > > > > > > > > >
> > > > > > > > > > On Fri, Jul 15, 2022 at 09:47:27AM +0530, Ani Sinha 
> > > > > > > > > > wrote:
> > > > > > > > > > > > Instead of all this mess, can't we just spawn e.g. 
> > > > > > > > > > "git clone --depth
> > > > > > > > > > 1"?
> > > > > > > > > > > > And if the directory exists I would fetch and 
> > > > > > > > > > checkout.
> > > > > > > > > > >
> > > > > > > > > > > There are two reasons I can think of why I do not 
> > > > > > > > > > like this idea:
> > > > > > > > > > >
> > > > > > > > > > > (a) a git clone of a whole directory would download 
> > > > > > > > > > all versions of the
> > > > > > > > > > > binary whereas we want only a specific version.
> > > > > > > > > >
> > > > > > > > > > You mention shallow clone yourself, and I used --depth 
> > > > > > > > > > 1 above.
> > > > > > > > > >
> > > > > > > > > > > Downloading a single file
> > > > > > > > > > > by shallow cloning or creating a git archive is 
> > > > > > > > > > overkill IMHO when a wget
> > > > > > > > > > > style retrieval works just fine.
> > > > > > > > > >
> > > > > > > > > > However, it does not provide for versioning, tagging 
> > > > > > > > > > etc so you have
> > > > > > > > > > to implement your own schema.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Hmm I’m not sure if we need all that. Bits has its own 
> > > > > > > > > > versioning mechanism and
> > > > > > > > > > I think all we need to do is maintain the same versioning 
> > > > > > > > > > logic and maintain
> > > > > > > > > > binaries of different  versions. Do we really need the 
> > > > > > > > > > power of git/version
> > > > > > > > > > control here? Dunno.
> > > > > > > > >
> > > > > > > > > Well we need some schema. Given we are not using official 
> > > > > > > > > bits releases
> > > > > > > > > I don't think we can reuse theirs.
> > > > > > > >
> > > > > > > > OK fine. Lets figuire out how to push bits somewhere in 
> > > > > > > > git.qemu.org and
> > > > > > > > the binaries in some other repo first. Everything else hinges 
> > > > > > > > on that. We
> > > > > > > > can fix the rest of the bits later incrementally.
> > > > > > >
> > > > > > > DanPB, any thoughts on putting bits on git.qemu.org or where and 
> > > > > > > how to
> > > > > > > keep the binaries?
> > > > > >
> > > > > > Can we please conclude on this?
> > > > > > Peter, can you please fork the repo? I have tried many times to 
> > > > > > reach
> > > > > > you on IRC but failed.
> > > > >
> > > > > Probably because of travel around KVM forum.
> > > > >
> > > > > I think given our CI is under pressure again due to gitlab free tier
> > > > > limits, tying binaries to CI isn't a great idea at this stage.
> > > > > Can Ani just upload binaies to qemu.org for now?
> > > >
> > > > I agree with Michael here. Having a full ci/cd job for this is
> > > > overkill IMHO. We should create a repo just for the binaries, have a
> > > > README there to explain how we generate them and check in new versions
> > > > as and when needed (it won't be frequent).
> > > > How about biosbits-bin repo?
> > >
> > > If QEMU is hosting binaries, where any part contains GPL code, then we
> > > need to be providing the full and corresponding source and the build
> > > scripts needed to re-create the binary. Once we have such scripts it
> > > should be trivial to trigger that from a CI job. If it isn't then
> > > we're doing something wrong.  The CI quota is not an issue, because
> > > this is not a job that we need to run continuously. It can be triggered
> > > manually as & when we decide we need to refresh the binary, so would
> > > be a small one-off CI quota hit.
> > >
> > > Also note that gitlab is intending to start enforcing storage quota
> > > on projects in the not too distant future. This makes it unappealing
> > > to store binaries in git repos, unless we genuinely need the ability
> > > to access historical versions of the binary. I don't believe we need
> > > that for biosbits.
> > >
> > > The

Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits

2022-09-28 Thread Ani Sinha

On Wed, Sep 28, 2022 at 9:28 AM Michael S. Tsirkin  wrote:
>
> On Wed, Sep 28, 2022 at 08:38:54AM +0530, Ani Sinha wrote:
> > > I don't really care where we upload them but only having the
> > > latest version is just going to break anything expecting
> > > the old binary.
> >
> > In fairness, I am not entirely certain if there is a tight coupling
> > between the qemu tests and the bits binaries. I have written the test
> > framework in a way such that test modifications and new tests can be
> > pushed into the bits binaries and the iso gets regenerated with the
> > new tests from QEMU itself before running the tests. Only when we need
> > bits bugfixes or say upgrade to new acpica that we would need to
> > regenerate the bits binaries.
>
> Theoretically, that's correct. But if we did not have bugs we would
> not need tests.

Hmm, you might have a point. Curious, do we keep versioned binaries of
edk for example? If so, why we can't do the same for bits?

Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits

On Wed, Sep 28, 2022 at 01:10:15AM +0200, Paolo Bonzini wrote:
> 
> 
> Il mer 28 set 2022, 00:09 Michael S. Tsirkin  ha scritto:
> 
> On Tue, Sep 27, 2022 at 11:44:56PM +0200, Paolo Bonzini wrote:
> > I also second the idea of using avocado instead of pytest, by the way.
> >
> > Paolo
> 
> I do not think this is a good fit for bios tests.
> bios tests are intended for a wide audience of ACPI developers
> across a variety of host systems. They basically do not need anything
> from the host and they need to be super easy to configure
> since we have lots of drive through contributors.
> 
> 
> The setup would be the same, with avocado installed in a virtual environment
> via pip. It doesn't need to be set up outside, neither with distro packages 
> nor
> in ~/.local, and especially it is not necessary to deal with avocado-vt.
> 
> Paolo

Hmm, good point.

-- 
MST

Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits

On Wed, Sep 28, 2022 at 08:38:54AM +0530, Ani Sinha wrote:
> > I don't really care where we upload them but only having the
> > latest version is just going to break anything expecting
> > the old binary.
> 
> In fairness, I am not entirely certain if there is a tight coupling
> between the qemu tests and the bits binaries. I have written the test
> framework in a way such that test modifications and new tests can be
> pushed into the bits binaries and the iso gets regenerated with the
> new tests from QEMU itself before running the tests. Only when we need
> bits bugfixes or say upgrade to new acpica that we would need to
> regenerate the bits binaries.

Theoretically, that's correct. But if we did not have bugs we would
not need tests.

-- 
MST

Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits

On Wed, Sep 28, 2022 at 2:48 AM Michael S. Tsirkin  wrote:
>
> On Tue, Sep 27, 2022 at 09:33:27AM +0100, Daniel P. Berrangé wrote:
> > On Tue, Sep 27, 2022 at 01:43:15PM +0530, Ani Sinha wrote:
> > > On Sun, Sep 18, 2022 at 1:58 AM Michael S. Tsirkin  
> > > wrote:
> > > >
> > > > On Fri, Sep 16, 2022 at 09:30:42PM +0530, Ani Sinha wrote:
> > > > > On Thu, Jul 28, 2022 at 12:08 AM Ani Sinha  wrote:
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Mon, 25 Jul 2022, Ani Sinha wrote:
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Sat, 16 Jul 2022, Michael S. Tsirkin wrote:
> > > > > > >
> > > > > > > > On Sat, Jul 16, 2022 at 12:06:00PM +0530, Ani Sinha wrote:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Fri, Jul 15, 2022 at 11:20 Michael S. Tsirkin 
> > > > > > > > >  wrote:
> > > > > > > > >
> > > > > > > > > On Fri, Jul 15, 2022 at 09:47:27AM +0530, Ani Sinha wrote:
> > > > > > > > > > > Instead of all this mess, can't we just spawn e.g. 
> > > > > > > > > "git clone --depth
> > > > > > > > > 1"?
> > > > > > > > > > > And if the directory exists I would fetch and 
> > > > > > > > > checkout.
> > > > > > > > > >
> > > > > > > > > > There are two reasons I can think of why I do not like 
> > > > > > > > > this idea:
> > > > > > > > > >
> > > > > > > > > > (a) a git clone of a whole directory would download all 
> > > > > > > > > versions of the
> > > > > > > > > > binary whereas we want only a specific version.
> > > > > > > > >
> > > > > > > > > You mention shallow clone yourself, and I used --depth 1 
> > > > > > > > > above.
> > > > > > > > >
> > > > > > > > > > Downloading a single file
> > > > > > > > > > by shallow cloning or creating a git archive is 
> > > > > > > > > overkill IMHO when a wget
> > > > > > > > > > style retrieval works just fine.
> > > > > > > > >
> > > > > > > > > However, it does not provide for versioning, tagging etc 
> > > > > > > > > so you have
> > > > > > > > > to implement your own schema.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Hmm I’m not sure if we need all that. Bits has its own 
> > > > > > > > > versioning mechanism and
> > > > > > > > > I think all we need to do is maintain the same versioning 
> > > > > > > > > logic and maintain
> > > > > > > > > binaries of different  versions. Do we really need the power 
> > > > > > > > > of git/version
> > > > > > > > > control here? Dunno.
> > > > > > > >
> > > > > > > > Well we need some schema. Given we are not using official bits 
> > > > > > > > releases
> > > > > > > > I don't think we can reuse theirs.
> > > > > > >
> > > > > > > OK fine. Lets figuire out how to push bits somewhere in 
> > > > > > > git.qemu.org and
> > > > > > > the binaries in some other repo first. Everything else hinges on 
> > > > > > > that. We
> > > > > > > can fix the rest of the bits later incrementally.
> > > > > >
> > > > > > DanPB, any thoughts on putting bits on git.qemu.org or where and 
> > > > > > how to
> > > > > > keep the binaries?
> > > > >
> > > > > Can we please conclude on this?
> > > > > Peter, can you please fork the repo? I have tried many times to reach
> > > > > you on IRC but failed.
> > > >
> > > > Probably because of travel around KVM forum.
> > > >
> > > > I think given our CI is under pressure again due to gitlab free tier
> > > > limits, tying binaries to CI isn't a great idea at this stage.
> > > > Can Ani just upload binaies to qemu.org for now?
> > >
> > > I agree with Michael here. Having a full ci/cd job for this is
> > > overkill IMHO. We should create a repo just for the binaries, have a
> > > README there to explain how we generate them and check in new versions
> > > as and when needed (it won't be frequent).
> > > How about biosbits-bin repo?
> >
> > If QEMU is hosting binaries, where any part contains GPL code, then we
> > need to be providing the full and corresponding source and the build
> > scripts needed to re-create the binary. Once we have such scripts it
> > should be trivial to trigger that from a CI job. If it isn't then
> > we're doing something wrong.  The CI quota is not an issue, because
> > this is not a job that we need to run continuously. It can be triggered
> > manually as & when we decide we need to refresh the binary, so would
> > be a small one-off CI quota hit.
> >
> > Also note that gitlab is intending to start enforcing storage quota
> > on projects in the not too distant future. This makes it unappealing
> > to store binaries in git repos, unless we genuinely need the ability
> > to access historical versions of the binary. I don't believe we need
> > that for biosbits.
> >
> > The binary can be published as a CI artifact and accessed directly
> > from the latest artifact download link. This ensures we only consume
> > quota for the most recently published binary artifact. So I don't see
> > a compelling reason to upload binaries into git.
> >
> > With regards,
> > Daniel
>
> I don't

Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits

2022-09-27 Thread Paolo Bonzini

Il mer 28 set 2022, 00:09 Michael S. Tsirkin  ha scritto:

> On Tue, Sep 27, 2022 at 11:44:56PM +0200, Paolo Bonzini wrote:
> > I also second the idea of using avocado instead of pytest, by the way.
> >
> > Paolo
>
> I do not think this is a good fit for bios tests.
> bios tests are intended for a wide audience of ACPI developers
> across a variety of host systems. They basically do not need anything
> from the host and they need to be super easy to configure
> since we have lots of drive through contributors.
>

The setup would be the same, with avocado installed in a virtual
environment via pip. It doesn't need to be set up outside, neither with
distro packages nor in ~/.local, and especially it is not necessary to deal
with avocado-vt.

Paolo

Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits

On Tue, Sep 27, 2022 at 11:44:56PM +0200, Paolo Bonzini wrote:
> I also second the idea of using avocado instead of pytest, by the way.
> 
> Paolo

I do not think this is a good fit for bios tests.
bios tests are intended for a wide audience of ACPI developers
across a variety of host systems. They basically do not need anything
from the host and they need to be super easy to configure
since we have lots of drive through contributors.

Problem is I don't think avocado is yet at the level where I can
ask random developers to use it to check their ACPI patches.

I just went ahead and rechecked and the situation isn't much better
yet. I think the focus of avocado is system testing of full guests with
KVM, not unit testing of ACPI.

Let's start with installation on a clean box:

following
https://avocado-framework.readthedocs.io/en/98.0/guides/user/chapters/installing.html

Ugh pip, will install a bunch of stuff in ~/.local and ask me to tweak
PATH ... and what about security? No thanks!

So ...
do I want LTS or latest? Well I donnu  let's try LTS?

$ dnf module enable avocado:82lts
[sudo] password for mst: 
Last metadata expiration check: 6 days, 15:20:21 ago on Wed 21 Sep 2022 
02:33:31 AM EDT.
Dependencies resolved.
==
 Package  ArchitectureVersion   
   RepositorySize
==
Enabling module streams:
 avocado  82lts 

Transaction Summary
==

Is this ok [y/N]: y
Complete!
[mst@tuck linux]$  dnf module install avocado
Last metadata expiration check: 6 days, 15:20:41 ago on Wed 21 Sep 2022 
02:33:31 AM EDT.
No default profiles for module avocado:82lts. Available profiles: default, 
minimal
Error: Problems in request:
broken groups or modules: avocado

Ugh I guess latest then?

[mst@tuck linux]$ dnf module enable avocado:latest
Last metadata expiration check: 6 days, 15:25:21 ago on Wed 21 Sep 2022 
02:33:31 AM EDT.
Dependencies resolved.
The operation would result in switching of module 'avocado' stream '82lts' to 
stream 'latest'
Error: It is not possible to switch enabled streams of a module unless 
explicitly enabled via configuration option module_stream_switch.
It is recommended to rather remove all installed content from the module, and 
reset the module using 'dnf module reset ' command. After you 
reset the module, you can install the other stream.

Scary ... I don't really know what are streams and I am guessing module
is avocado here? and what will this affect. Oh well, I'll risk this:

[mst@tuck linux]$ dnf module reset  avocado
Last metadata expiration check: 6 days, 15:25:46 ago on Wed 21 Sep 2022 
02:33:31 AM EDT.
Dependencies resolved.
==
 Package  ArchitectureVersion   
   RepositorySize
==
Resetting modules:
 avocado

Transaction Summary
==

Is this ok [y/N]: y
Complete!
[mst@tuck linux]$ dnf module enable avocado:latest
Last metadata expiration check: 6 days, 15:25:55 ago on Wed 21 Sep 2022 
02:33:31 AM EDT.
Dependencies resolved.
==
 Package  ArchitectureVersion   
   RepositorySize
==
Enabling module streams:
 avocado  latest

Transaction Summary
==

Is this ok [y/N]: y
Complete!
[mst@tuck linux]$  dnf module install avocado
Last metadata expiration check:

Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits

2022-09-27 Thread Paolo Bonzini

Il dom 10 lug 2022, 19:01 Ani Sinha  ha scritto:

> This change adds python based test environment that can be used to run
> pytest
> from within a virtual environment. A bash script sets up a virtual
> environment
> and then runs the python based tests from within that environment.
> All dependent python packages are installed in the virtual environment
> using
> pip python module. QEMU python test modules are also available in the
> environment
> for spawning the QEMU based VMs.
>
> It also introduces QEMU acpi/smbios biosbits python test script which is
> run
> from within the python virtual environment. When the bios bits tests are
> run,
> bios bits binaries are downloaded from an external repo/location.
> Currently, the test points to an external private github repo where the
> bits
> archives are checked in.


The virtual environment should be set up from configure, similar to git
submodules. John was working on it and probably can point you at some
earlier discussions in the archives about how to do it.

I also second the idea of using avocado instead of pytest, by the way.

Paolo


> Signed-off-by: Ani Sinha 
> ---
>  tests/pytest/acpi-bits/acpi-bits-test-venv.sh |  59 +++
>  tests/pytest/acpi-bits/acpi-bits-test.py  | 382 ++
>  tests/pytest/acpi-bits/meson.build|  33 ++
>  tests/pytest/acpi-bits/requirements.txt   |   1 +
>  4 files changed, 475 insertions(+)
>  create mode 100644 tests/pytest/acpi-bits/acpi-bits-test-venv.sh
>  create mode 100644 tests/pytest/acpi-bits/acpi-bits-test.py
>  create mode 100644 tests/pytest/acpi-bits/meson.build
>  create mode 100644 tests/pytest/acpi-bits/requirements.txt
>
> diff --git a/tests/pytest/acpi-bits/acpi-bits-test-venv.sh
> b/tests/pytest/acpi-bits/acpi-bits-test-venv.sh
> new file mode 100644
> index 00..186395473b
> --- /dev/null
> +++ b/tests/pytest/acpi-bits/acpi-bits-test-venv.sh
> @@ -0,0 +1,59 @@
> +#!/usr/bin/env bash
> +# Generates a python virtual environment for the test to run.
> +# Then runs python test scripts from within that virtual environment.
> +#
> +# This program is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License as published by
> +# the Free Software Foundation; either version 2 of the License, or
> +# (at your option) any later version.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program.  If not, see .
> +#
> +# Author: Ani Sinha 
> +
> +set -e
> +
> +MYPATH=$(realpath ${BASH_SOURCE:-$0})
> +MYDIR=$(dirname $MYPATH)
> +
> +if [ -z "$PYTEST_SOURCE_ROOT" ]; then
> +echo -n "Please set QTEST_SOURCE_ROOT env pointing"
> +echo " to the root of the qemu source tree."
> +echo -n "This is required so that the test can find the "
> +echo "python modules that it needs for execution."
> +exit 1
> +fi
> +SRCDIR=$PYTEST_SOURCE_ROOT
> +TESTSCRIPTS=("acpi-bits-test.py")
> +PIPCMD="-m pip -q --disable-pip-version-check"
> +# we need to save the old value of PWD before we do a change-dir later
> +PYTEST_PWD=$PWD
> +
> +TESTS_PYTHON=/usr/bin/python3
> +TESTS_VENV_REQ=requirements.txt
> +
> +# sadly for pip -e and -t options do not work together.
> +# please see https://github.com/pypa/pip/issues/562
> +cd $MYDIR
> +
> +$TESTS_PYTHON -m venv .
> +$TESTS_PYTHON $PIPCMD install -e $SRCDIR/python/
> +[ -f $TESTS_VENV_REQ ] && \
> +$TESTS_PYTHON $PIPCMD install -r $TESTS_VENV_REQ || exit 0
> +
> +# venv is activated at this point.
> +
> +# run the test
> +for testscript in ${TESTSCRIPTS[@]} ; do
> +export PYTEST_PWD; python3 $testscript
> +done
> +
> +cd $PYTEST_PWD
> +
> +exit 0
> diff --git a/tests/pytest/acpi-bits/acpi-bits-test.py
> b/tests/pytest/acpi-bits/acpi-bits-test.py
> new file mode 100644
> index 00..97e61eb709
> --- /dev/null
> +++ b/tests/pytest/acpi-bits/acpi-bits-test.py
> @@ -0,0 +1,382 @@
> +#!/usr/bin/env python3
> +# group: rw quick
> +# Exercize QEMU generated ACPI/SMBIOS tables using biosbits,
> +# https://biosbits.org/
> +#
> +# This program is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License as published by
> +# the Free Software Foundation; either version 2 of the License, or
> +# (at your option) any later version.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program.  If not, see

Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits

On Tue, Sep 27, 2022 at 03:37:39PM +0530, Ani Sinha wrote:
> > > > > > >
> > > > > > > OK fine. Lets figuire out how to push bits somewhere in 
> > > > > > > git.qemu.org and
> > > > > > > the binaries in some other repo first. Everything else hinges on 
> > > > > > > that. We
> > > > > > > can fix the rest of the bits later incrementally.
> > > > > >
> > > > > > DanPB, any thoughts on putting bits on git.qemu.org or where and 
> > > > > > how to
> > > > > > keep the binaries?
> > > > >
> > > > > Can we please conclude on this?
> > > > > Peter, can you please fork the repo? I have tried many times to reach
> > > > > you on IRC but failed.
> > > >
> > > > Probably because of travel around KVM forum.
> > > >
> > > > I think given our CI is under pressure again due to gitlab free tier
> > > > limits, tying binaries to CI isn't a great idea at this stage.
> > > > Can Ani just upload binaies to qemu.org for now?
> > >
> > > I agree with Michael here. Having a full ci/cd job for this is
> > > overkill IMHO. We should create a repo just for the binaries, have a
> > > README there to explain how we generate them and check in new versions
> > > as and when needed (it won't be frequent).
> > > How about biosbits-bin repo?
> >
> > If QEMU is hosting binaries, where any part contains GPL code, then we
> > need to be providing the full and corresponding source and the build
> > scripts needed to re-create the binary. Once we have such scripts it
> > should be trivial to trigger that from a CI job. If it isn't then
> > we're doing something wrong.
> 
> I was thinking of commiting the build scripts in a branch of
> https://gitlab.com/qemu-project/biosbits-bits.
> This would separate them from the main source. The scripts would build
> a version of qemu-bits based on the version information passed to it
> from the environment.
> Before I committed the scripts, I wanted to check whether we would
> want to do that or have a separate repo containing the binaries and
> the build scripts.
> Seems we want the former.

A separate repo is standard imho. Don't see any advantages to
abusing git branches like that.

> As for the gitlab-ci, I looked at the yaml file and the qemu ones
> looks quite complicated. Can someone help me generate one based on the
> build script here?
> https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh
> 
> > The CI quota is not an issue, because
> > this is not a job that we need to run continuously. It can be triggered
> > manually as & when we decide we need to refresh the binary, so would
> > be a small one-off CI quota hit.
> >
> > Also note that gitlab is intending to start enforcing storage quota
> > on projects in the not too distant future. This makes it unappealing
> > to store binaries in git repos, unless we genuinely need the ability
> > to access historical versions of the binary. I don't believe we need
> > that for biosbits.
> >
> > The binary can be published as a CI artifact and accessed directly
> > from the latest artifact download link. This ensures we only consume
> > quota for the most recently published binary artifact. So I don't see
> > a compelling reason to upload binaries into git.
> >
> > With regards,
> > Daniel
> > --
> > |: https://berrange.com  -o-https://www.flickr.com/photos/dberrange 
> > :|
> > |: https://libvirt.org -o-https://fstop138.berrange.com 
> > :|
> > |: https://entangle-photo.org-o-https://www.instagram.com/dberrange 
> > :|
> >

Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits

On Tue, Sep 27, 2022 at 04:45:09PM +0100, Daniel P. Berrangé wrote:
> On Tue, Sep 27, 2022 at 07:35:13PM +0530, Ani Sinha wrote:
> > On Tue, Sep 27, 2022 at 5:12 PM Ani Sinha  wrote:
> > >
> > > On Tue, Sep 27, 2022 at 3:48 PM Daniel P. Berrangé  
> > > wrote:
> > > >
> > > > On Tue, Sep 27, 2022 at 03:37:39PM +0530, Ani Sinha wrote:
> > > > > > > > > > >
> > > > > > > > > > > OK fine. Lets figuire out how to push bits somewhere in 
> > > > > > > > > > > git.qemu.org and
> > > > > > > > > > > the binaries in some other repo first. Everything else 
> > > > > > > > > > > hinges on that. We
> > > > > > > > > > > can fix the rest of the bits later incrementally.
> > > > > > > > > >
> > > > > > > > > > DanPB, any thoughts on putting bits on git.qemu.org or 
> > > > > > > > > > where and how to
> > > > > > > > > > keep the binaries?
> > > > > > > > >
> > > > > > > > > Can we please conclude on this?
> > > > > > > > > Peter, can you please fork the repo? I have tried many times 
> > > > > > > > > to reach
> > > > > > > > > you on IRC but failed.
> > > > > > > >
> > > > > > > > Probably because of travel around KVM forum.
> > > > > > > >
> > > > > > > > I think given our CI is under pressure again due to gitlab free 
> > > > > > > > tier
> > > > > > > > limits, tying binaries to CI isn't a great idea at this stage.
> > > > > > > > Can Ani just upload binaies to qemu.org for now?
> > > > > > >
> > > > > > > I agree with Michael here. Having a full ci/cd job for this is
> > > > > > > overkill IMHO. We should create a repo just for the binaries, 
> > > > > > > have a
> > > > > > > README there to explain how we generate them and check in new 
> > > > > > > versions
> > > > > > > as and when needed (it won't be frequent).
> > > > > > > How about biosbits-bin repo?
> > > > > >
> > > > > > If QEMU is hosting binaries, where any part contains GPL code, then 
> > > > > > we
> > > > > > need to be providing the full and corresponding source and the build
> > > > > > scripts needed to re-create the binary. Once we have such scripts it
> > > > > > should be trivial to trigger that from a CI job. If it isn't then
> > > > > > we're doing something wrong.
> > > > >
> > > > > I was thinking of commiting the build scripts in a branch of
> > > > > https://gitlab.com/qemu-project/biosbits-bits.
> > > > > This would separate them from the main source. The scripts would build
> > > > > a version of qemu-bits based on the version information passed to it
> > > > > from the environment.
> > > > > Before I committed the scripts, I wanted to check whether we would
> > > > > want to do that or have a separate repo containing the binaries and
> > > > > the build scripts.
> > > > > Seems we want the former.
> > > > >
> > > > > As for the gitlab-ci, I looked at the yaml file and the qemu ones
> > > > > looks quite complicated. Can someone help me generate one based on the
> > > > > build script here?
> > > > > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh
> > > >
> > > > Yes, QEMU's rules aren't a good place to start if you're trying
> > > > to learn gitlab CI, as they're very advanced.
> > > >
> > > > The simple case though is quite simple.
> > > >
> > > >   * You need a container image to act as the build env
> > > >   * In 'before_script'  install any packages you need on top of the
> > > > base container image for build deps
> > > >   * In 'script'  run whatever shell commands you need in order
> > > > to build the project
> > > >   * Add an 'artifacts' section to publish one (or more) files/dirs
> > > > as output
> > > >
> > > > The simplest example would be something like
> > > >
> > > >mybuild:
> > > >  image: fedora:36
> > > >  before_script:
> > > >- dnf install -y gcc
> > > >  script:
> > > >- gcc -o myapp myapp.c
> > > >  artifacts
> > > >paths:
> > > >  - myapp
> > > >
> > >
> > > How does this look?
> > > https://pastebin.com/0YyLFmi3
> > 
> > Alright, .gitlab-ci.yml is produced and the pipeline succeeds.
> > However, the question still remains, where do we keep the generated
> > artifacts?
> 
> The following link will always reflect the published artifacts from
> the most recently fully successful CI pipeline, on the 'qemu-bits'
> branch, and 'qemu-bits-build' CI job:
> 
> https://gitlab.com/qemu-project/biosbits-bits/-/jobs/artifacts/qemu-bits/download?job=qemu-bits-build
> 
> Tweak as needed if you push the CI to master branch instead. This
> link can be considered the permanent home of the artifact. I'd just
> suggest that the QEMU job automatically skip if it fails to download
> the artifact, as occassionally transient infra errors can impact
> it.
> 
> With regards,
> Daniel

This just means once we change the test old qemu source can no longer use it.
Why is this a good idea? Are we so short on disk space? I thought CPU
is the limiting factor?

> -- 
> |: https://berrange.com  -o-https://www.flickr.com/photos/dberrange

Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits

On Tue, Sep 27, 2022 at 09:33:27AM +0100, Daniel P. Berrangé wrote:
> On Tue, Sep 27, 2022 at 01:43:15PM +0530, Ani Sinha wrote:
> > On Sun, Sep 18, 2022 at 1:58 AM Michael S. Tsirkin  wrote:
> > >
> > > On Fri, Sep 16, 2022 at 09:30:42PM +0530, Ani Sinha wrote:
> > > > On Thu, Jul 28, 2022 at 12:08 AM Ani Sinha  wrote:
> > > > >
> > > > >
> > > > >
> > > > > On Mon, 25 Jul 2022, Ani Sinha wrote:
> > > > >
> > > > > >
> > > > > >
> > > > > > On Sat, 16 Jul 2022, Michael S. Tsirkin wrote:
> > > > > >
> > > > > > > On Sat, Jul 16, 2022 at 12:06:00PM +0530, Ani Sinha wrote:
> > > > > > > >
> > > > > > > >
> > > > > > > > On Fri, Jul 15, 2022 at 11:20 Michael S. Tsirkin 
> > > > > > > >  wrote:
> > > > > > > >
> > > > > > > > On Fri, Jul 15, 2022 at 09:47:27AM +0530, Ani Sinha wrote:
> > > > > > > > > > Instead of all this mess, can't we just spawn e.g. "git 
> > > > > > > > clone --depth
> > > > > > > > 1"?
> > > > > > > > > > And if the directory exists I would fetch and checkout.
> > > > > > > > >
> > > > > > > > > There are two reasons I can think of why I do not like 
> > > > > > > > this idea:
> > > > > > > > >
> > > > > > > > > (a) a git clone of a whole directory would download all 
> > > > > > > > versions of the
> > > > > > > > > binary whereas we want only a specific version.
> > > > > > > >
> > > > > > > > You mention shallow clone yourself, and I used --depth 1 
> > > > > > > > above.
> > > > > > > >
> > > > > > > > > Downloading a single file
> > > > > > > > > by shallow cloning or creating a git archive is overkill 
> > > > > > > > IMHO when a wget
> > > > > > > > > style retrieval works just fine.
> > > > > > > >
> > > > > > > > However, it does not provide for versioning, tagging etc so 
> > > > > > > > you have
> > > > > > > > to implement your own schema.
> > > > > > > >
> > > > > > > >
> > > > > > > > Hmm I’m not sure if we need all that. Bits has its own 
> > > > > > > > versioning mechanism and
> > > > > > > > I think all we need to do is maintain the same versioning logic 
> > > > > > > > and maintain
> > > > > > > > binaries of different  versions. Do we really need the power of 
> > > > > > > > git/version
> > > > > > > > control here? Dunno.
> > > > > > >
> > > > > > > Well we need some schema. Given we are not using official bits 
> > > > > > > releases
> > > > > > > I don't think we can reuse theirs.
> > > > > >
> > > > > > OK fine. Lets figuire out how to push bits somewhere in 
> > > > > > git.qemu.org and
> > > > > > the binaries in some other repo first. Everything else hinges on 
> > > > > > that. We
> > > > > > can fix the rest of the bits later incrementally.
> > > > >
> > > > > DanPB, any thoughts on putting bits on git.qemu.org or where and how 
> > > > > to
> > > > > keep the binaries?
> > > >
> > > > Can we please conclude on this?
> > > > Peter, can you please fork the repo? I have tried many times to reach
> > > > you on IRC but failed.
> > >
> > > Probably because of travel around KVM forum.
> > >
> > > I think given our CI is under pressure again due to gitlab free tier
> > > limits, tying binaries to CI isn't a great idea at this stage.
> > > Can Ani just upload binaies to qemu.org for now?
> > 
> > I agree with Michael here. Having a full ci/cd job for this is
> > overkill IMHO. We should create a repo just for the binaries, have a
> > README there to explain how we generate them and check in new versions
> > as and when needed (it won't be frequent).
> > How about biosbits-bin repo?
> 
> If QEMU is hosting binaries, where any part contains GPL code, then we
> need to be providing the full and corresponding source and the build
> scripts needed to re-create the binary. Once we have such scripts it
> should be trivial to trigger that from a CI job. If it isn't then
> we're doing something wrong.  The CI quota is not an issue, because
> this is not a job that we need to run continuously. It can be triggered
> manually as & when we decide we need to refresh the binary, so would
> be a small one-off CI quota hit.
> 
> Also note that gitlab is intending to start enforcing storage quota
> on projects in the not too distant future. This makes it unappealing
> to store binaries in git repos, unless we genuinely need the ability
> to access historical versions of the binary. I don't believe we need
> that for biosbits.
> 
> The binary can be published as a CI artifact and accessed directly
> from the latest artifact download link. This ensures we only consume
> quota for the most recently published binary artifact. So I don't see
> a compelling reason to upload binaries into git.
> 
> With regards,
> Daniel

I don't really care where we upload them but only having the
latest version is just going to break anything expecting
the old binary.



> -- 
> |: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org -o-https://fstop138.berrange.com :|
> |:

Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits

2022-09-27 Thread Daniel P . Berrangé

On Tue, Sep 27, 2022 at 07:35:13PM +0530, Ani Sinha wrote:
> On Tue, Sep 27, 2022 at 5:12 PM Ani Sinha  wrote:
> >
> > On Tue, Sep 27, 2022 at 3:48 PM Daniel P. Berrangé  
> > wrote:
> > >
> > > On Tue, Sep 27, 2022 at 03:37:39PM +0530, Ani Sinha wrote:
> > > > > > > > > >
> > > > > > > > > > OK fine. Lets figuire out how to push bits somewhere in 
> > > > > > > > > > git.qemu.org and
> > > > > > > > > > the binaries in some other repo first. Everything else 
> > > > > > > > > > hinges on that. We
> > > > > > > > > > can fix the rest of the bits later incrementally.
> > > > > > > > >
> > > > > > > > > DanPB, any thoughts on putting bits on git.qemu.org or where 
> > > > > > > > > and how to
> > > > > > > > > keep the binaries?
> > > > > > > >
> > > > > > > > Can we please conclude on this?
> > > > > > > > Peter, can you please fork the repo? I have tried many times to 
> > > > > > > > reach
> > > > > > > > you on IRC but failed.
> > > > > > >
> > > > > > > Probably because of travel around KVM forum.
> > > > > > >
> > > > > > > I think given our CI is under pressure again due to gitlab free 
> > > > > > > tier
> > > > > > > limits, tying binaries to CI isn't a great idea at this stage.
> > > > > > > Can Ani just upload binaies to qemu.org for now?
> > > > > >
> > > > > > I agree with Michael here. Having a full ci/cd job for this is
> > > > > > overkill IMHO. We should create a repo just for the binaries, have a
> > > > > > README there to explain how we generate them and check in new 
> > > > > > versions
> > > > > > as and when needed (it won't be frequent).
> > > > > > How about biosbits-bin repo?
> > > > >
> > > > > If QEMU is hosting binaries, where any part contains GPL code, then we
> > > > > need to be providing the full and corresponding source and the build
> > > > > scripts needed to re-create the binary. Once we have such scripts it
> > > > > should be trivial to trigger that from a CI job. If it isn't then
> > > > > we're doing something wrong.
> > > >
> > > > I was thinking of commiting the build scripts in a branch of
> > > > https://gitlab.com/qemu-project/biosbits-bits.
> > > > This would separate them from the main source. The scripts would build
> > > > a version of qemu-bits based on the version information passed to it
> > > > from the environment.
> > > > Before I committed the scripts, I wanted to check whether we would
> > > > want to do that or have a separate repo containing the binaries and
> > > > the build scripts.
> > > > Seems we want the former.
> > > >
> > > > As for the gitlab-ci, I looked at the yaml file and the qemu ones
> > > > looks quite complicated. Can someone help me generate one based on the
> > > > build script here?
> > > > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh
> > >
> > > Yes, QEMU's rules aren't a good place to start if you're trying
> > > to learn gitlab CI, as they're very advanced.
> > >
> > > The simple case though is quite simple.
> > >
> > >   * You need a container image to act as the build env
> > >   * In 'before_script'  install any packages you need on top of the
> > > base container image for build deps
> > >   * In 'script'  run whatever shell commands you need in order
> > > to build the project
> > >   * Add an 'artifacts' section to publish one (or more) files/dirs
> > > as output
> > >
> > > The simplest example would be something like
> > >
> > >mybuild:
> > >  image: fedora:36
> > >  before_script:
> > >- dnf install -y gcc
> > >  script:
> > >- gcc -o myapp myapp.c
> > >  artifacts
> > >paths:
> > >  - myapp
> > >
> >
> > How does this look?
> > https://pastebin.com/0YyLFmi3
> 
> Alright, .gitlab-ci.yml is produced and the pipeline succeeds.
> However, the question still remains, where do we keep the generated
> artifacts?

The following link will always reflect the published artifacts from
the most recently fully successful CI pipeline, on the 'qemu-bits'
branch, and 'qemu-bits-build' CI job:

https://gitlab.com/qemu-project/biosbits-bits/-/jobs/artifacts/qemu-bits/download?job=qemu-bits-build

Tweak as needed if you push the CI to master branch instead. This
link can be considered the permanent home of the artifact. I'd just
suggest that the QEMU job automatically skip if it fails to download
the artifact, as occassionally transient infra errors can impact
it.

With regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits

On Tue, Sep 27, 2022 at 5:12 PM Ani Sinha  wrote:
>
> On Tue, Sep 27, 2022 at 3:48 PM Daniel P. Berrangé  
> wrote:
> >
> > On Tue, Sep 27, 2022 at 03:37:39PM +0530, Ani Sinha wrote:
> > > > > > > > >
> > > > > > > > > OK fine. Lets figuire out how to push bits somewhere in 
> > > > > > > > > git.qemu.org and
> > > > > > > > > the binaries in some other repo first. Everything else hinges 
> > > > > > > > > on that. We
> > > > > > > > > can fix the rest of the bits later incrementally.
> > > > > > > >
> > > > > > > > DanPB, any thoughts on putting bits on git.qemu.org or where 
> > > > > > > > and how to
> > > > > > > > keep the binaries?
> > > > > > >
> > > > > > > Can we please conclude on this?
> > > > > > > Peter, can you please fork the repo? I have tried many times to 
> > > > > > > reach
> > > > > > > you on IRC but failed.
> > > > > >
> > > > > > Probably because of travel around KVM forum.
> > > > > >
> > > > > > I think given our CI is under pressure again due to gitlab free tier
> > > > > > limits, tying binaries to CI isn't a great idea at this stage.
> > > > > > Can Ani just upload binaies to qemu.org for now?
> > > > >
> > > > > I agree with Michael here. Having a full ci/cd job for this is
> > > > > overkill IMHO. We should create a repo just for the binaries, have a
> > > > > README there to explain how we generate them and check in new versions
> > > > > as and when needed (it won't be frequent).
> > > > > How about biosbits-bin repo?
> > > >
> > > > If QEMU is hosting binaries, where any part contains GPL code, then we
> > > > need to be providing the full and corresponding source and the build
> > > > scripts needed to re-create the binary. Once we have such scripts it
> > > > should be trivial to trigger that from a CI job. If it isn't then
> > > > we're doing something wrong.
> > >
> > > I was thinking of commiting the build scripts in a branch of
> > > https://gitlab.com/qemu-project/biosbits-bits.
> > > This would separate them from the main source. The scripts would build
> > > a version of qemu-bits based on the version information passed to it
> > > from the environment.
> > > Before I committed the scripts, I wanted to check whether we would
> > > want to do that or have a separate repo containing the binaries and
> > > the build scripts.
> > > Seems we want the former.
> > >
> > > As for the gitlab-ci, I looked at the yaml file and the qemu ones
> > > looks quite complicated. Can someone help me generate one based on the
> > > build script here?
> > > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh
> >
> > Yes, QEMU's rules aren't a good place to start if you're trying
> > to learn gitlab CI, as they're very advanced.
> >
> > The simple case though is quite simple.
> >
> >   * You need a container image to act as the build env
> >   * In 'before_script'  install any packages you need on top of the
> > base container image for build deps
> >   * In 'script'  run whatever shell commands you need in order
> > to build the project
> >   * Add an 'artifacts' section to publish one (or more) files/dirs
> > as output
> >
> > The simplest example would be something like
> >
> >mybuild:
> >  image: fedora:36
> >  before_script:
> >- dnf install -y gcc
> >  script:
> >- gcc -o myapp myapp.c
> >  artifacts
> >paths:
> >  - myapp
> >
>
> How does this look?
> https://pastebin.com/0YyLFmi3

Alright, .gitlab-ci.yml is produced and the pipeline succeeds.
However, the question still remains, where do we keep the generated
artifacts?

Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits

On Tue, Sep 27, 2022 at 3:48 PM Daniel P. Berrangé  wrote:
>
> On Tue, Sep 27, 2022 at 03:37:39PM +0530, Ani Sinha wrote:
> > > > > > > >
> > > > > > > > OK fine. Lets figuire out how to push bits somewhere in 
> > > > > > > > git.qemu.org and
> > > > > > > > the binaries in some other repo first. Everything else hinges 
> > > > > > > > on that. We
> > > > > > > > can fix the rest of the bits later incrementally.
> > > > > > >
> > > > > > > DanPB, any thoughts on putting bits on git.qemu.org or where and 
> > > > > > > how to
> > > > > > > keep the binaries?
> > > > > >
> > > > > > Can we please conclude on this?
> > > > > > Peter, can you please fork the repo? I have tried many times to 
> > > > > > reach
> > > > > > you on IRC but failed.
> > > > >
> > > > > Probably because of travel around KVM forum.
> > > > >
> > > > > I think given our CI is under pressure again due to gitlab free tier
> > > > > limits, tying binaries to CI isn't a great idea at this stage.
> > > > > Can Ani just upload binaies to qemu.org for now?
> > > >
> > > > I agree with Michael here. Having a full ci/cd job for this is
> > > > overkill IMHO. We should create a repo just for the binaries, have a
> > > > README there to explain how we generate them and check in new versions
> > > > as and when needed (it won't be frequent).
> > > > How about biosbits-bin repo?
> > >
> > > If QEMU is hosting binaries, where any part contains GPL code, then we
> > > need to be providing the full and corresponding source and the build
> > > scripts needed to re-create the binary. Once we have such scripts it
> > > should be trivial to trigger that from a CI job. If it isn't then
> > > we're doing something wrong.
> >
> > I was thinking of commiting the build scripts in a branch of
> > https://gitlab.com/qemu-project/biosbits-bits.
> > This would separate them from the main source. The scripts would build
> > a version of qemu-bits based on the version information passed to it
> > from the environment.
> > Before I committed the scripts, I wanted to check whether we would
> > want to do that or have a separate repo containing the binaries and
> > the build scripts.
> > Seems we want the former.
> >
> > As for the gitlab-ci, I looked at the yaml file and the qemu ones
> > looks quite complicated. Can someone help me generate one based on the
> > build script here?
> > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh
>
> Yes, QEMU's rules aren't a good place to start if you're trying
> to learn gitlab CI, as they're very advanced.
>
> The simple case though is quite simple.
>
>   * You need a container image to act as the build env
>   * In 'before_script'  install any packages you need on top of the
> base container image for build deps
>   * In 'script'  run whatever shell commands you need in order
> to build the project
>   * Add an 'artifacts' section to publish one (or more) files/dirs
> as output
>
> The simplest example would be something like
>
>mybuild:
>  image: fedora:36
>  before_script:
>- dnf install -y gcc
>  script:
>- gcc -o myapp myapp.c
>  artifacts
>paths:
>  - myapp
>

How does this look?
https://pastebin.com/0YyLFmi3

>
> So essentially use 'docker run yourimage:tag' locally to bring
> up the build env, and interactively work out what packags yuo
> need to install, and what scripts to invoke to build it. When
> you have it figured out, just copy those commands into the
> before_script/script sections in .gitlab-ci.yml
>
>
> Note, the .gitlab-ci.yml file would be in the git repo of the
> project you're building typically, which would be the main
> 'biosbits' project i presume.
>
> There's a fairly decent reference of syntax here:
>
>   https://docs.gitlab.com/ee/ci/yaml/
>
> And broader set of docs
>
>  https://docs.gitlab.com/ee/ci/
>
> With regards,
> Daniel
> --
> |: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org -o-https://fstop138.berrange.com :|
> |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
>

Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits

On Tue, Sep 27, 2022 at 15:48 Daniel P. Berrangé 
wrote:

> On Tue, Sep 27, 2022 at 03:37:39PM +0530, Ani Sinha wrote:
> > > > > > > >
> > > > > > > > OK fine. Lets figuire out how to push bits somewhere in
> git.qemu.org and
> > > > > > > > the binaries in some other repo first. Everything else
> hinges on that. We
> > > > > > > > can fix the rest of the bits later incrementally.
> > > > > > >
> > > > > > > DanPB, any thoughts on putting bits on git.qemu.org or where
> and how to
> > > > > > > keep the binaries?
> > > > > >
> > > > > > Can we please conclude on this?
> > > > > > Peter, can you please fork the repo? I have tried many times to
> reach
> > > > > > you on IRC but failed.
> > > > >
> > > > > Probably because of travel around KVM forum.
> > > > >
> > > > > I think given our CI is under pressure again due to gitlab free
> tier
> > > > > limits, tying binaries to CI isn't a great idea at this stage.
> > > > > Can Ani just upload binaies to qemu.org for now?
> > > >
> > > > I agree with Michael here. Having a full ci/cd job for this is
> > > > overkill IMHO. We should create a repo just for the binaries, have a
> > > > README there to explain how we generate them and check in new
> versions
> > > > as and when needed (it won't be frequent).
> > > > How about biosbits-bin repo?
> > >
> > > If QEMU is hosting binaries, where any part contains GPL code, then we
> > > need to be providing the full and corresponding source and the build
> > > scripts needed to re-create the binary. Once we have such scripts it
> > > should be trivial to trigger that from a CI job. If it isn't then
> > > we're doing something wrong.
> >
> > I was thinking of commiting the build scripts in a branch of
> > https://gitlab.com/qemu-project/biosbits-bits.
> > This would separate them from the main source. The scripts would build
> > a version of qemu-bits based on the version information passed to it
> > from the environment.
> > Before I committed the scripts, I wanted to check whether we would
> > want to do that or have a separate repo containing the binaries and
> > the build scripts.
> > Seems we want the former.
> >
> > As for the gitlab-ci, I looked at the yaml file and the qemu ones
> > looks quite complicated. Can someone help me generate one based on the
> > build script here?
> >
> https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh
>
> Yes, QEMU's rules aren't a good place to start if you're trying
> to learn gitlab CI, as they're very advanced.
>
> The simple case though is quite simple.
>
>   * You need a container image to act as the build env
>   * In 'before_script'  install any packages you need on top of the
> base container image for build deps
>   * In 'script'  run whatever shell commands you need in order
> to build the project
>   * Add an 'artifacts' section to publish one (or more) files/dirs
> as output
>
> The simplest example would be something like
>
>mybuild:
>  image: fedora:36
>  before_script:
>- dnf install -y gcc
>  script:
>- gcc -o myapp myapp.c
>  artifacts
>paths:
>  - myapp
>
>
> So essentially use 'docker run yourimage:tag' locally to bring
> up the build env, and interactively work out what packags yuo
> need to install, and what scripts to invoke to build it. When
> you have it figured out, just copy those commands into the
> before_script/script sections in .gitlab-ci.yml


I already have a Dockerfile and a build script rahe uses it to generate the
binaries

https://github.com/ani-sinha/bits/blob/bits-qemu-logging/Dockerfile


>
>
> Note, the .gitlab-ci.yml file would be in the git repo of the
> project you're building typically, which would be the main
> 'biosbits' project i presume.
>
> There's a fairly decent reference of syntax here:
>
>   https://docs.gitlab.com/ee/ci/yaml/
>
> And broader set of docs
>
>  https://docs.gitlab.com/ee/ci/
>
> With regards,
> Daniel
> --
> |: https://berrange.com  -o-
> https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org -o-
> https://fstop138.berrange.com :|
> |: https://entangle-photo.org-o-
> https://www.instagram.com/dberrange :|
>
>

Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits

2022-09-27 Thread Daniel P . Berrangé

On Tue, Sep 27, 2022 at 03:37:39PM +0530, Ani Sinha wrote:
> > > > > > >
> > > > > > > OK fine. Lets figuire out how to push bits somewhere in 
> > > > > > > git.qemu.org and
> > > > > > > the binaries in some other repo first. Everything else hinges on 
> > > > > > > that. We
> > > > > > > can fix the rest of the bits later incrementally.
> > > > > >
> > > > > > DanPB, any thoughts on putting bits on git.qemu.org or where and 
> > > > > > how to
> > > > > > keep the binaries?
> > > > >
> > > > > Can we please conclude on this?
> > > > > Peter, can you please fork the repo? I have tried many times to reach
> > > > > you on IRC but failed.
> > > >
> > > > Probably because of travel around KVM forum.
> > > >
> > > > I think given our CI is under pressure again due to gitlab free tier
> > > > limits, tying binaries to CI isn't a great idea at this stage.
> > > > Can Ani just upload binaies to qemu.org for now?
> > >
> > > I agree with Michael here. Having a full ci/cd job for this is
> > > overkill IMHO. We should create a repo just for the binaries, have a
> > > README there to explain how we generate them and check in new versions
> > > as and when needed (it won't be frequent).
> > > How about biosbits-bin repo?
> >
> > If QEMU is hosting binaries, where any part contains GPL code, then we
> > need to be providing the full and corresponding source and the build
> > scripts needed to re-create the binary. Once we have such scripts it
> > should be trivial to trigger that from a CI job. If it isn't then
> > we're doing something wrong.
> 
> I was thinking of commiting the build scripts in a branch of
> https://gitlab.com/qemu-project/biosbits-bits.
> This would separate them from the main source. The scripts would build
> a version of qemu-bits based on the version information passed to it
> from the environment.
> Before I committed the scripts, I wanted to check whether we would
> want to do that or have a separate repo containing the binaries and
> the build scripts.
> Seems we want the former.
> 
> As for the gitlab-ci, I looked at the yaml file and the qemu ones
> looks quite complicated. Can someone help me generate one based on the
> build script here?
> https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh

Yes, QEMU's rules aren't a good place to start if you're trying
to learn gitlab CI, as they're very advanced.

The simple case though is quite simple.

  * You need a container image to act as the build env
  * In 'before_script'  install any packages you need on top of the
base container image for build deps
  * In 'script'  run whatever shell commands you need in order
to build the project
  * Add an 'artifacts' section to publish one (or more) files/dirs
as output

The simplest example would be something like

   mybuild:
 image: fedora:36
 before_script:
   - dnf install -y gcc
 script:
   - gcc -o myapp myapp.c
 artifacts
   paths:
 - myapp


So essentially use 'docker run yourimage:tag' locally to bring
up the build env, and interactively work out what packags yuo
need to install, and what scripts to invoke to build it. When
you have it figured out, just copy those commands into the
before_script/script sections in .gitlab-ci.yml


Note, the .gitlab-ci.yml file would be in the git repo of the
project you're building typically, which would be the main
'biosbits' project i presume. 

There's a fairly decent reference of syntax here:

  https://docs.gitlab.com/ee/ci/yaml/

And broader set of docs

 https://docs.gitlab.com/ee/ci/

With regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits

> > > > > >
> > > > > > OK fine. Lets figuire out how to push bits somewhere in 
> > > > > > git.qemu.org and
> > > > > > the binaries in some other repo first. Everything else hinges on 
> > > > > > that. We
> > > > > > can fix the rest of the bits later incrementally.
> > > > >
> > > > > DanPB, any thoughts on putting bits on git.qemu.org or where and how 
> > > > > to
> > > > > keep the binaries?
> > > >
> > > > Can we please conclude on this?
> > > > Peter, can you please fork the repo? I have tried many times to reach
> > > > you on IRC but failed.
> > >
> > > Probably because of travel around KVM forum.
> > >
> > > I think given our CI is under pressure again due to gitlab free tier
> > > limits, tying binaries to CI isn't a great idea at this stage.
> > > Can Ani just upload binaies to qemu.org for now?
> >
> > I agree with Michael here. Having a full ci/cd job for this is
> > overkill IMHO. We should create a repo just for the binaries, have a
> > README there to explain how we generate them and check in new versions
> > as and when needed (it won't be frequent).
> > How about biosbits-bin repo?
>
> If QEMU is hosting binaries, where any part contains GPL code, then we
> need to be providing the full and corresponding source and the build
> scripts needed to re-create the binary. Once we have such scripts it
> should be trivial to trigger that from a CI job. If it isn't then
> we're doing something wrong.

I was thinking of commiting the build scripts in a branch of
https://gitlab.com/qemu-project/biosbits-bits.
This would separate them from the main source. The scripts would build
a version of qemu-bits based on the version information passed to it
from the environment.
Before I committed the scripts, I wanted to check whether we would
want to do that or have a separate repo containing the binaries and
the build scripts.
Seems we want the former.

As for the gitlab-ci, I looked at the yaml file and the qemu ones
looks quite complicated. Can someone help me generate one based on the
build script here?
https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh

> The CI quota is not an issue, because
> this is not a job that we need to run continuously. It can be triggered
> manually as & when we decide we need to refresh the binary, so would
> be a small one-off CI quota hit.
>
> Also note that gitlab is intending to start enforcing storage quota
> on projects in the not too distant future. This makes it unappealing
> to store binaries in git repos, unless we genuinely need the ability
> to access historical versions of the binary. I don't believe we need
> that for biosbits.
>
> The binary can be published as a CI artifact and accessed directly
> from the latest artifact download link. This ensures we only consume
> quota for the most recently published binary artifact. So I don't see
> a compelling reason to upload binaries into git.
>
> With regards,
> Daniel
> --
> |: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org -o-https://fstop138.berrange.com :|
> |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
>

Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits

2022-09-27 Thread Daniel P . Berrangé

On Tue, Sep 27, 2022 at 01:43:15PM +0530, Ani Sinha wrote:
> On Sun, Sep 18, 2022 at 1:58 AM Michael S. Tsirkin  wrote:
> >
> > On Fri, Sep 16, 2022 at 09:30:42PM +0530, Ani Sinha wrote:
> > > On Thu, Jul 28, 2022 at 12:08 AM Ani Sinha  wrote:
> > > >
> > > >
> > > >
> > > > On Mon, 25 Jul 2022, Ani Sinha wrote:
> > > >
> > > > >
> > > > >
> > > > > On Sat, 16 Jul 2022, Michael S. Tsirkin wrote:
> > > > >
> > > > > > On Sat, Jul 16, 2022 at 12:06:00PM +0530, Ani Sinha wrote:
> > > > > > >
> > > > > > >
> > > > > > > On Fri, Jul 15, 2022 at 11:20 Michael S. Tsirkin 
> > > > > > >  wrote:
> > > > > > >
> > > > > > > On Fri, Jul 15, 2022 at 09:47:27AM +0530, Ani Sinha wrote:
> > > > > > > > > Instead of all this mess, can't we just spawn e.g. "git 
> > > > > > > clone --depth
> > > > > > > 1"?
> > > > > > > > > And if the directory exists I would fetch and checkout.
> > > > > > > >
> > > > > > > > There are two reasons I can think of why I do not like this 
> > > > > > > idea:
> > > > > > > >
> > > > > > > > (a) a git clone of a whole directory would download all 
> > > > > > > versions of the
> > > > > > > > binary whereas we want only a specific version.
> > > > > > >
> > > > > > > You mention shallow clone yourself, and I used --depth 1 
> > > > > > > above.
> > > > > > >
> > > > > > > > Downloading a single file
> > > > > > > > by shallow cloning or creating a git archive is overkill 
> > > > > > > IMHO when a wget
> > > > > > > > style retrieval works just fine.
> > > > > > >
> > > > > > > However, it does not provide for versioning, tagging etc so 
> > > > > > > you have
> > > > > > > to implement your own schema.
> > > > > > >
> > > > > > >
> > > > > > > Hmm I’m not sure if we need all that. Bits has its own versioning 
> > > > > > > mechanism and
> > > > > > > I think all we need to do is maintain the same versioning logic 
> > > > > > > and maintain
> > > > > > > binaries of different  versions. Do we really need the power of 
> > > > > > > git/version
> > > > > > > control here? Dunno.
> > > > > >
> > > > > > Well we need some schema. Given we are not using official bits 
> > > > > > releases
> > > > > > I don't think we can reuse theirs.
> > > > >
> > > > > OK fine. Lets figuire out how to push bits somewhere in git.qemu.org 
> > > > > and
> > > > > the binaries in some other repo first. Everything else hinges on 
> > > > > that. We
> > > > > can fix the rest of the bits later incrementally.
> > > >
> > > > DanPB, any thoughts on putting bits on git.qemu.org or where and how to
> > > > keep the binaries?
> > >
> > > Can we please conclude on this?
> > > Peter, can you please fork the repo? I have tried many times to reach
> > > you on IRC but failed.
> >
> > Probably because of travel around KVM forum.
> >
> > I think given our CI is under pressure again due to gitlab free tier
> > limits, tying binaries to CI isn't a great idea at this stage.
> > Can Ani just upload binaies to qemu.org for now?
> 
> I agree with Michael here. Having a full ci/cd job for this is
> overkill IMHO. We should create a repo just for the binaries, have a
> README there to explain how we generate them and check in new versions
> as and when needed (it won't be frequent).
> How about biosbits-bin repo?

If QEMU is hosting binaries, where any part contains GPL code, then we
need to be providing the full and corresponding source and the build
scripts needed to re-create the binary. Once we have such scripts it
should be trivial to trigger that from a CI job. If it isn't then
we're doing something wrong.  The CI quota is not an issue, because
this is not a job that we need to run continuously. It can be triggered
manually as & when we decide we need to refresh the binary, so would
be a small one-off CI quota hit.

Also note that gitlab is intending to start enforcing storage quota
on projects in the not too distant future. This makes it unappealing
to store binaries in git repos, unless we genuinely need the ability
to access historical versions of the binary. I don't believe we need
that for biosbits.

The binary can be published as a CI artifact and accessed directly
from the latest artifact download link. This ensures we only consume
quota for the most recently published binary artifact. So I don't see
a compelling reason to upload binaries into git.

With regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits