Re: [PATCH RFC 01/32] python/qemu: create qemu.lib module
On 6/3/20 5:00 AM, Kevin Wolf wrote: > Am 02.06.2020 um 18:44 hat John Snow geschrieben: >> >> >> On 6/2/20 6:08 AM, Kevin Wolf wrote: >>> Am 14.05.2020 um 07:53 hat John Snow geschrieben: move python/qemu/*.py to python/qemu/lib/*.py. To create a namespace package, the 'qemu' directory itself shouldn't have module files in it. Thus, these files will go under a 'lib' package directory instead. Bolster the lib/__init__.py file a little bit, Make the top-level classes and functions available directly inside the `qemu.lib` namespace, to facilitate a convenient shorthand: > from qemu.lib import QEMUQtestMachine, QEMUMonitorProtocol Lastly, update all of the existing import directives. (Note: these scripts were not necessarily tested to see if they still work. Some of these scripts are in obvious states of disrepair and it is beyond the scope of this patch to attempt to fix them.) Signed-off-by: John Snow --- python/qemu/__init__.py | 11 - >>> >>> When trying to reproduce your mypy problem, I was wondering why mypy was >>> complaining that it couldn't find qemu.lib. The reason is that removing >>> __init__.py from qemu means it's not a valid module any more. If I >>> recreate it locally, mypy stops complaining. >>> >>> So I think we need to leave this file here. >>> >>> Kevin >>> >> >> Depends. You'll want --namespace-packages to parse a PEP420 namespace. >> >> (It's not a given we definitely want a PEP420 namespace, but that's what >> I created here.) > > I'm just running 297, so if this is how mypy should be called now, the > series needs to make a change to 297. > > Kevin > Yes, sorry -- 297 wasn't merged yet when I wrote the 32 patch RFC, and my followup series to add mypy support is now factored to occur prior to the library split. I have corrected this in the more recent postings, sorry for the inconvenience.
Re: [PATCH RFC 01/32] python/qemu: create qemu.lib module
Am 02.06.2020 um 18:44 hat John Snow geschrieben: > > > On 6/2/20 6:08 AM, Kevin Wolf wrote: > > Am 14.05.2020 um 07:53 hat John Snow geschrieben: > >> move python/qemu/*.py to python/qemu/lib/*.py. > >> > >> To create a namespace package, the 'qemu' directory itself shouldn't > >> have module files in it. Thus, these files will go under a 'lib' package > >> directory instead. > >> > >> Bolster the lib/__init__.py file a little bit, Make the top-level > >> classes and functions available directly inside the `qemu.lib` > >> namespace, to facilitate a convenient shorthand: > >> > >>> from qemu.lib import QEMUQtestMachine, QEMUMonitorProtocol > >> > >> Lastly, update all of the existing import directives. > >> > >> (Note: these scripts were not necessarily tested to see if they still > >> work. Some of these scripts are in obvious states of disrepair and it is > >> beyond the scope of this patch to attempt to fix them.) > >> > >> Signed-off-by: John Snow > >> --- > >> python/qemu/__init__.py | 11 - > > > > When trying to reproduce your mypy problem, I was wondering why mypy was > > complaining that it couldn't find qemu.lib. The reason is that removing > > __init__.py from qemu means it's not a valid module any more. If I > > recreate it locally, mypy stops complaining. > > > > So I think we need to leave this file here. > > > > Kevin > > > > Depends. You'll want --namespace-packages to parse a PEP420 namespace. > > (It's not a given we definitely want a PEP420 namespace, but that's what > I created here.) I'm just running 297, so if this is how mypy should be called now, the series needs to make a change to 297. Kevin
Re: [PATCH RFC 01/32] python/qemu: create qemu.lib module
On 6/2/20 6:08 AM, Kevin Wolf wrote: > Am 14.05.2020 um 07:53 hat John Snow geschrieben: >> move python/qemu/*.py to python/qemu/lib/*.py. >> >> To create a namespace package, the 'qemu' directory itself shouldn't >> have module files in it. Thus, these files will go under a 'lib' package >> directory instead. >> >> Bolster the lib/__init__.py file a little bit, Make the top-level >> classes and functions available directly inside the `qemu.lib` >> namespace, to facilitate a convenient shorthand: >> >>> from qemu.lib import QEMUQtestMachine, QEMUMonitorProtocol >> >> Lastly, update all of the existing import directives. >> >> (Note: these scripts were not necessarily tested to see if they still >> work. Some of these scripts are in obvious states of disrepair and it is >> beyond the scope of this patch to attempt to fix them.) >> >> Signed-off-by: John Snow >> --- >> python/qemu/__init__.py | 11 - > > When trying to reproduce your mypy problem, I was wondering why mypy was > complaining that it couldn't find qemu.lib. The reason is that removing > __init__.py from qemu means it's not a valid module any more. If I > recreate it locally, mypy stops complaining. > > So I think we need to leave this file here. > > Kevin > Depends. You'll want --namespace-packages to parse a PEP420 namespace. (It's not a given we definitely want a PEP420 namespace, but that's what I created here.) --js
Re: [PATCH RFC 01/32] python/qemu: create qemu.lib module
26.05.2020 18:07, Philippe Mathieu-Daudé wrote: On 5/19/20 12:54 PM, Vladimir Sementsov-Ogievskiy wrote: 19.05.2020 03:27, John Snow wrote: On 5/18/20 3:33 PM, Vladimir Sementsov-Ogievskiy wrote: 18.05.2020 21:23, John Snow wrote: On 5/18/20 2:14 PM, Vladimir Sementsov-Ogievskiy wrote: 14.05.2020 08:53, John Snow wrote: move python/qemu/*.py to python/qemu/lib/*.py. To create a namespace package, the 'qemu' directory itself shouldn't have module files in it. Thus, these files will go under a 'lib' package directory instead. Hmm.. On the first glance, it looks better to have from qemu import QEMUMachine than from qemu.lib import QEMUMachine why do we need this extra ".lib" part? Is it needed only for internal use? Assume we have installed qemu package. Can we write from qemu import QEMUMachine ? Or we still need qemu.lib ? I don't remember any python package, which made me to write "import from package_name.lib ..." It's a strategy to create "qemu" as a PEP420 namespace package; i.e. "qemu" forms a namespace, but you need a name for the actual package underneath it. "qemu.lib" is one package, with qmp, qtest, and machine modules. "qemu" isn't really a package in this system, it's just a namespace. The idea is that this allows us to create a more modular rollout of various python scripts and services as desired instead of monolithically bundling them all inside of a "qemu" package. It also allows us to fork or split out the sub-packages to separate repos, if we wish. i.e., let's say we create a "qemu.sdk" subpackage, we can eventually fork it off into its own repo with its own installer and so forth. These subpackages can be installed and managed separately. Okay, I understand.. No real objections than. Still, maybe, everything should not go into lib, maybe something like qemu/vm/ - qmp, QEMUMachine, etc qemu/qtest/ - qtest I'm not sure this part is relevant now, as we have not good projection of what/who/how this package will be consumed. I suppose by VM you mean VirtualMachine. I find it confusing. Maybe simply "machine"? We also have 'tools' and 'user-space processes'. QMP is protocol, common to all. "qemu.core.qmp"? We also have the gdb(stub) protocol, common to machine(system) & user. The block layer has its classes, "qemu.block"? Sounds good. But I see now that we shouldn't care too much about this, as we just don't know how it all will be used.. would be more user friendly? But I'm not sure. I just thought that "lib" is too generic. lib is a very generic name, I agree. Splitting accel, qmp and QEMUMachine in one package and keeping qtest in another is fine too. I'm not sure if I like "vm" for the name of that core package, though. I want to avoid using "qemu/sdk" because I have some plans for trying to generate and package a "real" SDK using that namespace. "devkit"? "testkit"? "core"? Naming things is always the worst part. I think, "core" sounds good. Agreed. -- Best regards, Vladimir
Re: [PATCH RFC 01/32] python/qemu: create qemu.lib module
Am 14.05.2020 um 07:53 hat John Snow geschrieben: > move python/qemu/*.py to python/qemu/lib/*.py. > > To create a namespace package, the 'qemu' directory itself shouldn't > have module files in it. Thus, these files will go under a 'lib' package > directory instead. > > Bolster the lib/__init__.py file a little bit, Make the top-level > classes and functions available directly inside the `qemu.lib` > namespace, to facilitate a convenient shorthand: > > > from qemu.lib import QEMUQtestMachine, QEMUMonitorProtocol > > Lastly, update all of the existing import directives. > > (Note: these scripts were not necessarily tested to see if they still > work. Some of these scripts are in obvious states of disrepair and it is > beyond the scope of this patch to attempt to fix them.) > > Signed-off-by: John Snow > --- > python/qemu/__init__.py | 11 - When trying to reproduce your mypy problem, I was wondering why mypy was complaining that it couldn't find qemu.lib. The reason is that removing __init__.py from qemu means it's not a valid module any more. If I recreate it locally, mypy stops complaining. So I think we need to leave this file here. Kevin
Re: [PATCH RFC 01/32] python/qemu: create qemu.lib module
On Wed, May 27, 2020 at 10:28:44AM -0400, John Snow wrote: > > > On 5/26/20 11:25 AM, Daniel P. Berrangé wrote: > > On Tue, May 26, 2020 at 05:23:42PM +0200, Philippe Mathieu-Daudé wrote: > >> On 5/26/20 5:22 PM, Daniel P. Berrangé wrote: > >>> On Mon, May 18, 2020 at 08:27:54PM -0400, John Snow wrote: > > > On 5/18/20 3:33 PM, Vladimir Sementsov-Ogievskiy wrote: > > 18.05.2020 21:23, John Snow wrote: > >> > >> > >> On 5/18/20 2:14 PM, Vladimir Sementsov-Ogievskiy wrote: > >>> 14.05.2020 08:53, John Snow wrote: > move python/qemu/*.py to python/qemu/lib/*.py. > > To create a namespace package, the 'qemu' directory itself shouldn't > have module files in it. Thus, these files will go under a 'lib' > package > directory instead. > >>> > >>> Hmm.. > >>> > >>> On the first glance, it looks better to have > >>> > >>> from qemu import QEMUMachine > >>> > >>> than > >>> from qemu.lib import QEMUMachine > >>> > >>> why do we need this extra ".lib" part? > >>> > >>> Is it needed only for internal use? > >>> > >>> Assume we have installed qemu package. Can we write > >>> > >>> from qemu import QEMUMachine > >>> > >>> ? Or we still need qemu.lib ? > >>> > >>> I don't remember any python package, which made me to write "import > >>> from > >>> package_name.lib ..." > >>> > >>> > >> > >> It's a strategy to create "qemu" as a PEP420 namespace package; i.e. > >> "qemu" forms a namespace, but you need a name for the actual package > >> underneath it. > >> > >> "qemu.lib" is one package, with qmp, qtest, and machine modules. "qemu" > >> isn't really a package in this system, it's just a namespace. > >> > >> The idea is that this allows us to create a more modular rollout of > >> various python scripts and services as desired instead of > >> monolithically > >> bundling them all inside of a "qemu" package. > >> > >> It also allows us to fork or split out the sub-packages to separate > >> repos, if we wish. i.e., let's say we create a "qemu.sdk" subpackage, > >> we > >> can eventually fork it off into its own repo with its own installer and > >> so forth. These subpackages can be installed and managed separately. > >> > > > > Okay, I understand.. No real objections than. > > > > Still, maybe, everything should not go into lib, maybe something like > > > > qemu/vm/ - qmp, QEMUMachine, etc > > qemu/qtest/ - qtest > > > > would be more user friendly? But I'm not sure. I just thought that "lib" > > is too generic. > > > > lib is a very generic name, I agree. > > Splitting accel, qmp and QEMUMachine in one package and keeping qtest in > another is fine too. I'm not sure if I like "vm" for the name of that > core package, though. > > I want to avoid using "qemu/sdk" because I have some plans for trying to > generate and package a "real" SDK using that namespace. > > "devkit"? "testkit"? "core"? Naming things is always the worst part. > >>> > >>> I'd suggest "machine", as in > >>> > >>> from qemu.machine import kvm_available, QEMUMachine > >>> > >>> I wouldn't over-think the module naming as it has so little impact on > >>> the code usage - it usually only appears in the "import" statement. > >> > >> Don't forget linux-user binaries. > > > > That's why I suggested ".machine", as all the APIs there currently > > are focused on the machine emulators, and the linx-user binaries > > share essentially nothing in common with softmmu binaries in terms > > of control APIs / CLI config. We can add a "qemu.user" package > > later if we have stuff related to that to expose > > > I'm re-ordering the series to front-load the linting and type-checking; > and the package organization will now come second, in a separate series. > > Module naming isn't a big deal right now, but if we package it and > upload to PyPI it will be something we shouldn't change frivolously. > > Daniel, are you suggesting we split it like this? -- > > - qemu.machine (machine.py, qtest.py, accel.py?) > - qemu.monitor (qmp.py) I was actually suggesting everything in qemu.machine, but I guess qemu.monitor makes sense, given that this is an interface both for controlling QEMU and the guest agent. > the only one that's really truly weird is accel.py?, which is just kind > of a misc function. I guess it can go in `qemu.machine` for now and if > we adopt a `qemu.user` later, we can pull it out into a common area if > we need to. Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
Re: [PATCH RFC 01/32] python/qemu: create qemu.lib module
On 5/26/20 11:25 AM, Daniel P. Berrangé wrote: > On Tue, May 26, 2020 at 05:23:42PM +0200, Philippe Mathieu-Daudé wrote: >> On 5/26/20 5:22 PM, Daniel P. Berrangé wrote: >>> On Mon, May 18, 2020 at 08:27:54PM -0400, John Snow wrote: On 5/18/20 3:33 PM, Vladimir Sementsov-Ogievskiy wrote: > 18.05.2020 21:23, John Snow wrote: >> >> >> On 5/18/20 2:14 PM, Vladimir Sementsov-Ogievskiy wrote: >>> 14.05.2020 08:53, John Snow wrote: move python/qemu/*.py to python/qemu/lib/*.py. To create a namespace package, the 'qemu' directory itself shouldn't have module files in it. Thus, these files will go under a 'lib' package directory instead. >>> >>> Hmm.. >>> >>> On the first glance, it looks better to have >>> >>> from qemu import QEMUMachine >>> >>> than >>> from qemu.lib import QEMUMachine >>> >>> why do we need this extra ".lib" part? >>> >>> Is it needed only for internal use? >>> >>> Assume we have installed qemu package. Can we write >>> >>> from qemu import QEMUMachine >>> >>> ? Or we still need qemu.lib ? >>> >>> I don't remember any python package, which made me to write "import from >>> package_name.lib ..." >>> >>> >> >> It's a strategy to create "qemu" as a PEP420 namespace package; i.e. >> "qemu" forms a namespace, but you need a name for the actual package >> underneath it. >> >> "qemu.lib" is one package, with qmp, qtest, and machine modules. "qemu" >> isn't really a package in this system, it's just a namespace. >> >> The idea is that this allows us to create a more modular rollout of >> various python scripts and services as desired instead of monolithically >> bundling them all inside of a "qemu" package. >> >> It also allows us to fork or split out the sub-packages to separate >> repos, if we wish. i.e., let's say we create a "qemu.sdk" subpackage, we >> can eventually fork it off into its own repo with its own installer and >> so forth. These subpackages can be installed and managed separately. >> > > Okay, I understand.. No real objections than. > > Still, maybe, everything should not go into lib, maybe something like > > qemu/vm/ - qmp, QEMUMachine, etc > qemu/qtest/ - qtest > > would be more user friendly? But I'm not sure. I just thought that "lib" > is too generic. > lib is a very generic name, I agree. Splitting accel, qmp and QEMUMachine in one package and keeping qtest in another is fine too. I'm not sure if I like "vm" for the name of that core package, though. I want to avoid using "qemu/sdk" because I have some plans for trying to generate and package a "real" SDK using that namespace. "devkit"? "testkit"? "core"? Naming things is always the worst part. >>> >>> I'd suggest "machine", as in >>> >>> from qemu.machine import kvm_available, QEMUMachine >>> >>> I wouldn't over-think the module naming as it has so little impact on >>> the code usage - it usually only appears in the "import" statement. >> >> Don't forget linux-user binaries. > > That's why I suggested ".machine", as all the APIs there currently > are focused on the machine emulators, and the linx-user binaries > share essentially nothing in common with softmmu binaries in terms > of control APIs / CLI config. We can add a "qemu.user" package > later if we have stuff related to that to expose > I'm re-ordering the series to front-load the linting and type-checking; and the package organization will now come second, in a separate series. Module naming isn't a big deal right now, but if we package it and upload to PyPI it will be something we shouldn't change frivolously. Daniel, are you suggesting we split it like this? -- - qemu.machine (machine.py, qtest.py, accel.py?) - qemu.monitor (qmp.py) the only one that's really truly weird is accel.py?, which is just kind of a misc function. I guess it can go in `qemu.machine` for now and if we adopt a `qemu.user` later, we can pull it out into a common area if we need to. --js
Re: [PATCH RFC 01/32] python/qemu: create qemu.lib module
On Tue, May 26, 2020 at 05:23:42PM +0200, Philippe Mathieu-Daudé wrote: > On 5/26/20 5:22 PM, Daniel P. Berrangé wrote: > > On Mon, May 18, 2020 at 08:27:54PM -0400, John Snow wrote: > >> > >> > >> On 5/18/20 3:33 PM, Vladimir Sementsov-Ogievskiy wrote: > >>> 18.05.2020 21:23, John Snow wrote: > > > On 5/18/20 2:14 PM, Vladimir Sementsov-Ogievskiy wrote: > > 14.05.2020 08:53, John Snow wrote: > >> move python/qemu/*.py to python/qemu/lib/*.py. > >> > >> To create a namespace package, the 'qemu' directory itself shouldn't > >> have module files in it. Thus, these files will go under a 'lib' > >> package > >> directory instead. > > > > Hmm.. > > > > On the first glance, it looks better to have > > > > from qemu import QEMUMachine > > > > than > > from qemu.lib import QEMUMachine > > > > why do we need this extra ".lib" part? > > > > Is it needed only for internal use? > > > > Assume we have installed qemu package. Can we write > > > > from qemu import QEMUMachine > > > > ? Or we still need qemu.lib ? > > > > I don't remember any python package, which made me to write "import from > > package_name.lib ..." > > > > > > It's a strategy to create "qemu" as a PEP420 namespace package; i.e. > "qemu" forms a namespace, but you need a name for the actual package > underneath it. > > "qemu.lib" is one package, with qmp, qtest, and machine modules. "qemu" > isn't really a package in this system, it's just a namespace. > > The idea is that this allows us to create a more modular rollout of > various python scripts and services as desired instead of monolithically > bundling them all inside of a "qemu" package. > > It also allows us to fork or split out the sub-packages to separate > repos, if we wish. i.e., let's say we create a "qemu.sdk" subpackage, we > can eventually fork it off into its own repo with its own installer and > so forth. These subpackages can be installed and managed separately. > > >>> > >>> Okay, I understand.. No real objections than. > >>> > >>> Still, maybe, everything should not go into lib, maybe something like > >>> > >>> qemu/vm/ - qmp, QEMUMachine, etc > >>> qemu/qtest/ - qtest > >>> > >>> would be more user friendly? But I'm not sure. I just thought that "lib" > >>> is too generic. > >>> > >> > >> lib is a very generic name, I agree. > >> > >> Splitting accel, qmp and QEMUMachine in one package and keeping qtest in > >> another is fine too. I'm not sure if I like "vm" for the name of that > >> core package, though. > >> > >> I want to avoid using "qemu/sdk" because I have some plans for trying to > >> generate and package a "real" SDK using that namespace. > >> > >> "devkit"? "testkit"? "core"? Naming things is always the worst part. > > > > I'd suggest "machine", as in > > > > from qemu.machine import kvm_available, QEMUMachine > > > > I wouldn't over-think the module naming as it has so little impact on > > the code usage - it usually only appears in the "import" statement. > > Don't forget linux-user binaries. That's why I suggested ".machine", as all the APIs there currently are focused on the machine emulators, and the linx-user binaries share essentially nothing in common with softmmu binaries in terms of control APIs / CLI config. We can add a "qemu.user" package later if we have stuff related to that to expose Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
Re: [PATCH RFC 01/32] python/qemu: create qemu.lib module
On 5/26/20 5:22 PM, Daniel P. Berrangé wrote: > On Mon, May 18, 2020 at 08:27:54PM -0400, John Snow wrote: >> >> >> On 5/18/20 3:33 PM, Vladimir Sementsov-Ogievskiy wrote: >>> 18.05.2020 21:23, John Snow wrote: On 5/18/20 2:14 PM, Vladimir Sementsov-Ogievskiy wrote: > 14.05.2020 08:53, John Snow wrote: >> move python/qemu/*.py to python/qemu/lib/*.py. >> >> To create a namespace package, the 'qemu' directory itself shouldn't >> have module files in it. Thus, these files will go under a 'lib' >> package >> directory instead. > > Hmm.. > > On the first glance, it looks better to have > > from qemu import QEMUMachine > > than > from qemu.lib import QEMUMachine > > why do we need this extra ".lib" part? > > Is it needed only for internal use? > > Assume we have installed qemu package. Can we write > > from qemu import QEMUMachine > > ? Or we still need qemu.lib ? > > I don't remember any python package, which made me to write "import from > package_name.lib ..." > > It's a strategy to create "qemu" as a PEP420 namespace package; i.e. "qemu" forms a namespace, but you need a name for the actual package underneath it. "qemu.lib" is one package, with qmp, qtest, and machine modules. "qemu" isn't really a package in this system, it's just a namespace. The idea is that this allows us to create a more modular rollout of various python scripts and services as desired instead of monolithically bundling them all inside of a "qemu" package. It also allows us to fork or split out the sub-packages to separate repos, if we wish. i.e., let's say we create a "qemu.sdk" subpackage, we can eventually fork it off into its own repo with its own installer and so forth. These subpackages can be installed and managed separately. >>> >>> Okay, I understand.. No real objections than. >>> >>> Still, maybe, everything should not go into lib, maybe something like >>> >>> qemu/vm/ - qmp, QEMUMachine, etc >>> qemu/qtest/ - qtest >>> >>> would be more user friendly? But I'm not sure. I just thought that "lib" >>> is too generic. >>> >> >> lib is a very generic name, I agree. >> >> Splitting accel, qmp and QEMUMachine in one package and keeping qtest in >> another is fine too. I'm not sure if I like "vm" for the name of that >> core package, though. >> >> I want to avoid using "qemu/sdk" because I have some plans for trying to >> generate and package a "real" SDK using that namespace. >> >> "devkit"? "testkit"? "core"? Naming things is always the worst part. > > I'd suggest "machine", as in > > from qemu.machine import kvm_available, QEMUMachine > > I wouldn't over-think the module naming as it has so little impact on > the code usage - it usually only appears in the "import" statement. Don't forget linux-user binaries. > > > Regards, > Daniel >
Re: [PATCH RFC 01/32] python/qemu: create qemu.lib module
On Mon, May 18, 2020 at 08:27:54PM -0400, John Snow wrote: > > > On 5/18/20 3:33 PM, Vladimir Sementsov-Ogievskiy wrote: > > 18.05.2020 21:23, John Snow wrote: > >> > >> > >> On 5/18/20 2:14 PM, Vladimir Sementsov-Ogievskiy wrote: > >>> 14.05.2020 08:53, John Snow wrote: > move python/qemu/*.py to python/qemu/lib/*.py. > > To create a namespace package, the 'qemu' directory itself shouldn't > have module files in it. Thus, these files will go under a 'lib' > package > directory instead. > >>> > >>> Hmm.. > >>> > >>> On the first glance, it looks better to have > >>> > >>> from qemu import QEMUMachine > >>> > >>> than > >>> from qemu.lib import QEMUMachine > >>> > >>> why do we need this extra ".lib" part? > >>> > >>> Is it needed only for internal use? > >>> > >>> Assume we have installed qemu package. Can we write > >>> > >>> from qemu import QEMUMachine > >>> > >>> ? Or we still need qemu.lib ? > >>> > >>> I don't remember any python package, which made me to write "import from > >>> package_name.lib ..." > >>> > >>> > >> > >> It's a strategy to create "qemu" as a PEP420 namespace package; i.e. > >> "qemu" forms a namespace, but you need a name for the actual package > >> underneath it. > >> > >> "qemu.lib" is one package, with qmp, qtest, and machine modules. "qemu" > >> isn't really a package in this system, it's just a namespace. > >> > >> The idea is that this allows us to create a more modular rollout of > >> various python scripts and services as desired instead of monolithically > >> bundling them all inside of a "qemu" package. > >> > >> It also allows us to fork or split out the sub-packages to separate > >> repos, if we wish. i.e., let's say we create a "qemu.sdk" subpackage, we > >> can eventually fork it off into its own repo with its own installer and > >> so forth. These subpackages can be installed and managed separately. > >> > > > > Okay, I understand.. No real objections than. > > > > Still, maybe, everything should not go into lib, maybe something like > > > > qemu/vm/ - qmp, QEMUMachine, etc > > qemu/qtest/ - qtest > > > > would be more user friendly? But I'm not sure. I just thought that "lib" > > is too generic. > > > > lib is a very generic name, I agree. > > Splitting accel, qmp and QEMUMachine in one package and keeping qtest in > another is fine too. I'm not sure if I like "vm" for the name of that > core package, though. > > I want to avoid using "qemu/sdk" because I have some plans for trying to > generate and package a "real" SDK using that namespace. > > "devkit"? "testkit"? "core"? Naming things is always the worst part. I'd suggest "machine", as in from qemu.machine import kvm_available, QEMUMachine I wouldn't over-think the module naming as it has so little impact on the code usage - it usually only appears in the "import" statement. Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
Re: [PATCH RFC 01/32] python/qemu: create qemu.lib module
On 5/19/20 12:54 PM, Vladimir Sementsov-Ogievskiy wrote: > 19.05.2020 03:27, John Snow wrote: >> >> >> On 5/18/20 3:33 PM, Vladimir Sementsov-Ogievskiy wrote: >>> 18.05.2020 21:23, John Snow wrote: On 5/18/20 2:14 PM, Vladimir Sementsov-Ogievskiy wrote: > 14.05.2020 08:53, John Snow wrote: >> move python/qemu/*.py to python/qemu/lib/*.py. >> >> To create a namespace package, the 'qemu' directory itself shouldn't >> have module files in it. Thus, these files will go under a 'lib' >> package >> directory instead. > > Hmm.. > > On the first glance, it looks better to have > > from qemu import QEMUMachine > > than > from qemu.lib import QEMUMachine > > why do we need this extra ".lib" part? > > Is it needed only for internal use? > > Assume we have installed qemu package. Can we write > > from qemu import QEMUMachine > > ? Or we still need qemu.lib ? > > I don't remember any python package, which made me to write "import > from > package_name.lib ..." > > It's a strategy to create "qemu" as a PEP420 namespace package; i.e. "qemu" forms a namespace, but you need a name for the actual package underneath it. "qemu.lib" is one package, with qmp, qtest, and machine modules. "qemu" isn't really a package in this system, it's just a namespace. The idea is that this allows us to create a more modular rollout of various python scripts and services as desired instead of monolithically bundling them all inside of a "qemu" package. It also allows us to fork or split out the sub-packages to separate repos, if we wish. i.e., let's say we create a "qemu.sdk" subpackage, we can eventually fork it off into its own repo with its own installer and so forth. These subpackages can be installed and managed separately. >>> >>> Okay, I understand.. No real objections than. >>> >>> Still, maybe, everything should not go into lib, maybe something like >>> >>> qemu/vm/ - qmp, QEMUMachine, etc >>> qemu/qtest/ - qtest I'm not sure this part is relevant now, as we have not good projection of what/who/how this package will be consumed. I suppose by VM you mean VirtualMachine. I find it confusing. Maybe simply "machine"? We also have 'tools' and 'user-space processes'. QMP is protocol, common to all. "qemu.core.qmp"? We also have the gdb(stub) protocol, common to machine(system) & user. The block layer has its classes, "qemu.block"? >>> >>> would be more user friendly? But I'm not sure. I just thought that "lib" >>> is too generic. >>> >> >> lib is a very generic name, I agree. >> >> Splitting accel, qmp and QEMUMachine in one package and keeping qtest in >> another is fine too. I'm not sure if I like "vm" for the name of that >> core package, though. >> >> I want to avoid using "qemu/sdk" because I have some plans for trying to >> generate and package a "real" SDK using that namespace. >> >> "devkit"? "testkit"? "core"? Naming things is always the worst part. >> > > I think, "core" sounds good. Agreed.
Re: [PATCH RFC 01/32] python/qemu: create qemu.lib module
19.05.2020 03:27, John Snow wrote: On 5/18/20 3:33 PM, Vladimir Sementsov-Ogievskiy wrote: 18.05.2020 21:23, John Snow wrote: On 5/18/20 2:14 PM, Vladimir Sementsov-Ogievskiy wrote: 14.05.2020 08:53, John Snow wrote: move python/qemu/*.py to python/qemu/lib/*.py. To create a namespace package, the 'qemu' directory itself shouldn't have module files in it. Thus, these files will go under a 'lib' package directory instead. Hmm.. On the first glance, it looks better to have from qemu import QEMUMachine than from qemu.lib import QEMUMachine why do we need this extra ".lib" part? Is it needed only for internal use? Assume we have installed qemu package. Can we write from qemu import QEMUMachine ? Or we still need qemu.lib ? I don't remember any python package, which made me to write "import from package_name.lib ..." It's a strategy to create "qemu" as a PEP420 namespace package; i.e. "qemu" forms a namespace, but you need a name for the actual package underneath it. "qemu.lib" is one package, with qmp, qtest, and machine modules. "qemu" isn't really a package in this system, it's just a namespace. The idea is that this allows us to create a more modular rollout of various python scripts and services as desired instead of monolithically bundling them all inside of a "qemu" package. It also allows us to fork or split out the sub-packages to separate repos, if we wish. i.e., let's say we create a "qemu.sdk" subpackage, we can eventually fork it off into its own repo with its own installer and so forth. These subpackages can be installed and managed separately. Okay, I understand.. No real objections than. Still, maybe, everything should not go into lib, maybe something like qemu/vm/ - qmp, QEMUMachine, etc qemu/qtest/ - qtest would be more user friendly? But I'm not sure. I just thought that "lib" is too generic. lib is a very generic name, I agree. Splitting accel, qmp and QEMUMachine in one package and keeping qtest in another is fine too. I'm not sure if I like "vm" for the name of that core package, though. I want to avoid using "qemu/sdk" because I have some plans for trying to generate and package a "real" SDK using that namespace. "devkit"? "testkit"? "core"? Naming things is always the worst part. I think, "core" sounds good. -- Best regards, Vladimir
Re: [PATCH RFC 01/32] python/qemu: create qemu.lib module
On 5/18/20 3:33 PM, Vladimir Sementsov-Ogievskiy wrote: > 18.05.2020 21:23, John Snow wrote: >> >> >> On 5/18/20 2:14 PM, Vladimir Sementsov-Ogievskiy wrote: >>> 14.05.2020 08:53, John Snow wrote: move python/qemu/*.py to python/qemu/lib/*.py. To create a namespace package, the 'qemu' directory itself shouldn't have module files in it. Thus, these files will go under a 'lib' package directory instead. >>> >>> Hmm.. >>> >>> On the first glance, it looks better to have >>> >>> from qemu import QEMUMachine >>> >>> than >>> from qemu.lib import QEMUMachine >>> >>> why do we need this extra ".lib" part? >>> >>> Is it needed only for internal use? >>> >>> Assume we have installed qemu package. Can we write >>> >>> from qemu import QEMUMachine >>> >>> ? Or we still need qemu.lib ? >>> >>> I don't remember any python package, which made me to write "import from >>> package_name.lib ..." >>> >>> >> >> It's a strategy to create "qemu" as a PEP420 namespace package; i.e. >> "qemu" forms a namespace, but you need a name for the actual package >> underneath it. >> >> "qemu.lib" is one package, with qmp, qtest, and machine modules. "qemu" >> isn't really a package in this system, it's just a namespace. >> >> The idea is that this allows us to create a more modular rollout of >> various python scripts and services as desired instead of monolithically >> bundling them all inside of a "qemu" package. >> >> It also allows us to fork or split out the sub-packages to separate >> repos, if we wish. i.e., let's say we create a "qemu.sdk" subpackage, we >> can eventually fork it off into its own repo with its own installer and >> so forth. These subpackages can be installed and managed separately. >> > > Okay, I understand.. No real objections than. > > Still, maybe, everything should not go into lib, maybe something like > > qemu/vm/ - qmp, QEMUMachine, etc > qemu/qtest/ - qtest > > would be more user friendly? But I'm not sure. I just thought that "lib" > is too generic. > lib is a very generic name, I agree. Splitting accel, qmp and QEMUMachine in one package and keeping qtest in another is fine too. I'm not sure if I like "vm" for the name of that core package, though. I want to avoid using "qemu/sdk" because I have some plans for trying to generate and package a "real" SDK using that namespace. "devkit"? "testkit"? "core"? Naming things is always the worst part. --js
Re: [PATCH RFC 01/32] python/qemu: create qemu.lib module
18.05.2020 21:23, John Snow wrote: On 5/18/20 2:14 PM, Vladimir Sementsov-Ogievskiy wrote: 14.05.2020 08:53, John Snow wrote: move python/qemu/*.py to python/qemu/lib/*.py. To create a namespace package, the 'qemu' directory itself shouldn't have module files in it. Thus, these files will go under a 'lib' package directory instead. Hmm.. On the first glance, it looks better to have from qemu import QEMUMachine than from qemu.lib import QEMUMachine why do we need this extra ".lib" part? Is it needed only for internal use? Assume we have installed qemu package. Can we write from qemu import QEMUMachine ? Or we still need qemu.lib ? I don't remember any python package, which made me to write "import from package_name.lib ..." It's a strategy to create "qemu" as a PEP420 namespace package; i.e. "qemu" forms a namespace, but you need a name for the actual package underneath it. "qemu.lib" is one package, with qmp, qtest, and machine modules. "qemu" isn't really a package in this system, it's just a namespace. The idea is that this allows us to create a more modular rollout of various python scripts and services as desired instead of monolithically bundling them all inside of a "qemu" package. It also allows us to fork or split out the sub-packages to separate repos, if we wish. i.e., let's say we create a "qemu.sdk" subpackage, we can eventually fork it off into its own repo with its own installer and so forth. These subpackages can be installed and managed separately. Okay, I understand.. No real objections than. Still, maybe, everything should not go into lib, maybe something like qemu/vm/ - qmp, QEMUMachine, etc qemu/qtest/ - qtest would be more user friendly? But I'm not sure. I just thought that "lib" is too generic. -- Best regards, Vladimir
Re: [PATCH RFC 01/32] python/qemu: create qemu.lib module
On 5/18/20 2:14 PM, Vladimir Sementsov-Ogievskiy wrote: > 14.05.2020 08:53, John Snow wrote: >> move python/qemu/*.py to python/qemu/lib/*.py. >> >> To create a namespace package, the 'qemu' directory itself shouldn't >> have module files in it. Thus, these files will go under a 'lib' package >> directory instead. > > Hmm.. > > On the first glance, it looks better to have > > from qemu import QEMUMachine > > than > > from qemu.lib import QEMUMachine > > why do we need this extra ".lib" part? > > Is it needed only for internal use? > > Assume we have installed qemu package. Can we write > > from qemu import QEMUMachine > > ? Or we still need qemu.lib ? > > I don't remember any python package, which made me to write "import from > package_name.lib ..." > > It's a strategy to create "qemu" as a PEP420 namespace package; i.e. "qemu" forms a namespace, but you need a name for the actual package underneath it. "qemu.lib" is one package, with qmp, qtest, and machine modules. "qemu" isn't really a package in this system, it's just a namespace. The idea is that this allows us to create a more modular rollout of various python scripts and services as desired instead of monolithically bundling them all inside of a "qemu" package. It also allows us to fork or split out the sub-packages to separate repos, if we wish. i.e., let's say we create a "qemu.sdk" subpackage, we can eventually fork it off into its own repo with its own installer and so forth. These subpackages can be installed and managed separately.
Re: [PATCH RFC 01/32] python/qemu: create qemu.lib module
14.05.2020 08:53, John Snow wrote: move python/qemu/*.py to python/qemu/lib/*.py. To create a namespace package, the 'qemu' directory itself shouldn't have module files in it. Thus, these files will go under a 'lib' package directory instead. Hmm.. On the first glance, it looks better to have from qemu import QEMUMachine than from qemu.lib import QEMUMachine why do we need this extra ".lib" part? Is it needed only for internal use? Assume we have installed qemu package. Can we write from qemu import QEMUMachine ? Or we still need qemu.lib ? I don't remember any python package, which made me to write "import from package_name.lib ..." -- Best regards, Vladimir
[PATCH RFC 01/32] python/qemu: create qemu.lib module
move python/qemu/*.py to python/qemu/lib/*.py.
To create a namespace package, the 'qemu' directory itself shouldn't
have module files in it. Thus, these files will go under a 'lib' package
directory instead.
Bolster the lib/__init__.py file a little bit, Make the top-level
classes and functions available directly inside the `qemu.lib`
namespace, to facilitate a convenient shorthand:
> from qemu.lib import QEMUQtestMachine, QEMUMonitorProtocol
Lastly, update all of the existing import directives.
(Note: these scripts were not necessarily tested to see if they still
work. Some of these scripts are in obvious states of disrepair and it is
beyond the scope of this patch to attempt to fix them.)
Signed-off-by: John Snow
---
python/qemu/__init__.py | 11 -
python/qemu/lib/__init__.py | 57 +++
python/qemu/{ => lib}/accel.py| 0
python/qemu/{ => lib}/machine.py | 0
python/qemu/{ => lib}/qmp.py | 0
python/qemu/{ => lib}/qtest.py| 0
scripts/device-crash-test | 2 +-
scripts/qmp/qemu-ga-client| 2 +-
scripts/qmp/qmp-shell | 2 +-
scripts/render_block_graph.py | 3 +-
scripts/simplebench/bench_block_job.py| 4 +-
tests/acceptance/avocado_qemu/__init__.py | 2 +-
tests/acceptance/boot_linux.py| 3 +-
tests/acceptance/virtio_check_params.py | 2 +-
tests/acceptance/virtio_version.py| 2 +-
tests/migration/guestperf/engine.py | 2 +-
tests/qemu-iotests/235| 2 +-
tests/qemu-iotests/iotests.py | 2 +-
tests/vm/basevm.py| 6 +--
19 files changed, 74 insertions(+), 28 deletions(-)
delete mode 100644 python/qemu/__init__.py
create mode 100644 python/qemu/lib/__init__.py
rename python/qemu/{ => lib}/accel.py (100%)
rename python/qemu/{ => lib}/machine.py (100%)
rename python/qemu/{ => lib}/qmp.py (100%)
rename python/qemu/{ => lib}/qtest.py (100%)
diff --git a/python/qemu/__init__.py b/python/qemu/__init__.py
deleted file mode 100644
index 4ca06c34a4..00
--- a/python/qemu/__init__.py
+++ /dev/null
@@ -1,11 +0,0 @@
-# QEMU library
-#
-# Copyright (C) 2015-2016 Red Hat Inc.
-# Copyright (C) 2012 IBM Corp.
-#
-# Authors:
-# Fam Zheng
-#
-# This work is licensed under the terms of the GNU GPL, version 2. See
-# the COPYING file in the top-level directory.
-#
diff --git a/python/qemu/lib/__init__.py b/python/qemu/lib/__init__.py
new file mode 100644
index 00..afc587bfdc
--- /dev/null
+++ b/python/qemu/lib/__init__.py
@@ -0,0 +1,57 @@
+"""
+QEMU development and testing library.
+
+This library provides a few high-level classes for driving QEMU from a
+test suite, not intended for production use.
+
+- QEMUMachine: Configure and Boot a QEMU VM
+ - QEMUQtestMachine: VM class, with a qtest socket.
+
+- QEMUMonitorProtocol: Connect to, send/receive QMP messages.
+- QEMUQtestProtocol: Connect to, send/receive qtest message.
+
+- list_accel: List available accelerators
+- kvm_available: Probe for KVM support
+- tcg_available: Probe for TCG support
+"""
+
+# Copyright (C) 2020 John Snow for Red Hat Inc.
+# Copyright (C) 2015-2016 Red Hat Inc.
+# Copyright (C) 2012 IBM Corp.
+#
+# Authors:
+# John Snow
+# Fam Zheng
+#
+# This work is licensed under the terms of the GNU GPL, version 2. See
+# the COPYING file in the top-level directory.
+#
+
+from .accel import (
+list_accel,
+kvm_available,
+tcg_available,
+)
+
+from .qmp import (
+QEMUMonitorProtocol,
+)
+
+from .machine import (
+QEMUMachine,
+)
+
+from .qtest import (
+QEMUQtestProtocol,
+QEMUQtestMachine,
+)
+
+__all__ = (
+'list_accel',
+'kvm_available',
+'tcg_available',
+'QEMUMonitorProtocol',
+'QEMUMachine',
+'QEMUQtestProtocol',
+'QEMUQtestMachine',
+)
diff --git a/python/qemu/accel.py b/python/qemu/lib/accel.py
similarity index 100%
rename from python/qemu/accel.py
rename to python/qemu/lib/accel.py
diff --git a/python/qemu/machine.py b/python/qemu/lib/machine.py
similarity index 100%
rename from python/qemu/machine.py
rename to python/qemu/lib/machine.py
diff --git a/python/qemu/qmp.py b/python/qemu/lib/qmp.py
similarity index 100%
rename from python/qemu/qmp.py
rename to python/qemu/lib/qmp.py
diff --git a/python/qemu/qtest.py b/python/qemu/lib/qtest.py
similarity index 100%
rename from python/qemu/qtest.py
rename to python/qemu/lib/qtest.py
diff --git a/scripts/device-crash-test b/scripts/device-crash-test
index 305d0427af..49efd4abd7 100755
--- a/scripts/device-crash-test
+++ b/scripts/device-crash-test
@@ -35,7 +35,7 @@ import argparse
from itertools import chain
sys.path.append(os.path.join(os.path.dirname(__file__), '..', 'python'))
-from qemu.machine import QEMUMachine
+from qemu.lib import QEMUMachine
logger = logging.getLogger('device-crash-test')
dbg = logger.debug
diff --git
