Re: [vpp-dev] Python API modules
On Wed, Nov 18, 2020 at 4:38 AM wrote:
> > I think that generating the python code makes the most sense. The body
> of the code can still defer to the dynamically generated object, since it's
> not available until runtime anyway, but the user no longer has to resort to
> using lambdas because of the deferred bindings.
>
> Could you ellaborate for why you need to use lambdas?
>
Today, in the tests, all references to vpp_papi. have to happen after
connect, otherwise they don't exist. Let's say I want to set up a test and
use an enum from vpp_papi, it may not exist due to the deferred binding.
Instead, you have to wrap it in a lamba so that it is evaluated at use time
instead of at module load time. You could also make it a @property and
force it evaluated at run time.
> [SNIP]
> >
> > but the typedefs could become data classes and the encode/decode methods
> in the test code become unnecessary.
>
> What are you thinking about regarding the encode/decode in tests?
> E.g. for the IP address ones, they can just be deleted. Without any
> changes to PAPI...
>
> As an example:
class VppGbpContractNextHop():
def __init__(self, mac, bd, ip, rd):
self.mac = mac
self.ip = ip
self.bd = bd
self.rd = rd
def encode(self):
return {'ip': self.ip,
'mac': self.mac.packed,
'bd_id': self.bd.bd.bd_id,
'rd_id': self.rd.rd_id}
> > vl_api_ip_ecn_t.IP_API_ECN_NONE could become ip_ecn.NONE
> >
> > The IDE's could then provide type checking and auto completion. (So one
> would know that NONE was an option without referring to the sources.)
> >
> > We would need to play around with it to see how well it works.
>
> That seems like a good idea.
> Let me give you a quick Python plugin for vppapigen that does this.
>
> The packaging is going to be a challenge I guess.
>
> The packaging could be as simple as we write the files to a folder in
vpp_papi, or we could have its own setup.py and make it a plugin to
vpp_papi.
> Best regards,
> Ole
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18083): https://lists.fd.io/g/vpp-dev/message/18083
Mute This Topic: https://lists.fd.io/mt/78229638/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Python API modules
> I think that generating the python code makes the most sense. The body of > the code can still defer to the dynamically generated object, since it's not > available until runtime anyway, but the user no longer has to resort to using > lambdas because of the deferred bindings. Could you ellaborate for why you need to use lambdas? > I imagined it would work like a regular module: > > >>> import vpp_papi.plugins.map as map > >>> dir(map) > ... > >>> help(map) > > vppapigen would generate stubs such as: > > vpp_papi.plugins.map.py: > > def map_add_domain(client_index: u32, context: u32, ip6_prefix: IPV6Network, > ip4_prefix: IPV6Network, ip6_src: IPV6Interface, ea_bits_len : u8, > psid_offset : u8, psid_length : u8, mtu: u16, tag : str ) -> i32: >"""docstring from .api block comment >""" > > > > The user, of course, still needs to connect to VPP for their code to run. > > but the typedefs could become data classes and the encode/decode methods in > the test code become unnecessary. What are you thinking about regarding the encode/decode in tests? E.g. for the IP address ones, they can just be deleted. Without any changes to PAPI... > vl_api_ip_ecn_t.IP_API_ECN_NONE could become ip_ecn.NONE > > The IDE's could then provide type checking and auto completion. (So one would > know that NONE was an option without referring to the sources.) > > We would need to play around with it to see how well it works. That seems like a good idea. Let me give you a quick Python plugin for vppapigen that does this. The packaging is going to be a challenge I guess. Best regards, Ole -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#18079): https://lists.fd.io/g/vpp-dev/message/18079 Mute This Topic: https://lists.fd.io/mt/78229638/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Python API modules
I think that generating the python code makes the most sense. The body of the code can still defer to the dynamically generated object, since it's not available until runtime anyway, but the user no longer has to resort to using lambdas because of the deferred bindings. I imagined it would work like a regular module: >>> import vpp_papi.plugins.map as map >>> dir(map) ... >>> help(map) vppapigen would generate stubs such as: vpp_papi.plugins.map.py: def map_add_domain(client_index: u32, context: u32, ip6_prefix: IPV6Network, ip4_prefix: IPV6Network, ip6_src: IPV6Interface, ea_bits_len : u8, psid_offset : u8, psid_length : u8, mtu: u16, tag : str ) -> i32: """docstring from .api block comment """ > The user, of course, still needs to connect to VPP for their code to run. but the typedefs could become data classes and the encode/decode methods in the test code become unnecessary. vl_api_ip_ecn_t.IP_API_ECN_NONE could become ip_ecn.NONE The IDE's could then provide type checking and auto completion. (So one would know that NONE was an option without referring to the sources.) We would need to play around with it to see how well it works. On Sat, Nov 14, 2020 at 1:16 PM wrote: > Hi Paul, > > Picking off one issue at the time. > > > Why won't Ole, as maintainer, allow it? > > > > The build system uses something called vppapigen to generate the c > include files and the json files. It could as easily generate static stubs > so that development with python could be usable without a running vpp > instance. > > I certainly didn't intend for my opinion to come off as a "don't allow". > > Initially the Python code was fully generated from vppapigen. With a class > per .api file and fully formed methods. Later it evolved to dynamically > create all functions based on the API representations in JSON. > > (This latter change made the Python binding independent of VPP version > which is how it was possible to individually package it and put it on PyPI, > but lets do that on a separate thread.) > > Generating Python code from vppapigen is simple. But just how do you want > these stubs to look like and how should they be packaged and used? > > If I have pushed back on this it's likely more out of ignorance of the > problem rather than ill will. ;-) > > Best regards, > Ole -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#18065): https://lists.fd.io/g/vpp-dev/message/18065 Mute This Topic: https://lists.fd.io/mt/78229638/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Python API modules
> something has gone wrong here. Yeah, sorry for the confusion. I consult the maintainers file when double-checking who to add as reviewers. But when I think the maintainers file is outdated, I do not hesitate to add [0] or remove [1] anybody based on my (limited) perception. Usually I am too focused on fixing VPP to properly think about the process of editing the maintainers file. Do we have a document with guidelines related to that? Vratko. [0] https://gerrit.fd.io/r/c/vpp/+/20366 [1] https://gerrit.fd.io/r/c/vpp/+/22672 From: vpp-dev@lists.fd.io On Behalf Of Ole Troan Sent: Monday, 2020-November-16 15:16 To: Paul Vinciguerra Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) ; Marcos - Mgiga ; vpp-dev Subject: Re: [vpp-dev] Python API modules Hi Paul, > He/Vratko removed me as one of the maintainers of papi. > https://gerrit.fd.io/r/c/vpp/+/22672. I'm cool with not being a maintainer, > kinda funny that it was stuffed into another changeset. I have been called > out repeatedly for submitting unrelated changes ;) My apologies, something has gone wrong here. I wasn't even aware that you got removed. I have reviewed the history and spoken to Vratko to understand what has happened here. Vratko was editing the MAINTAINERS file and made changes in good faith reflecting what he believed was the current state. Expecting reviewers to comment. I missed that in code review, and the patch went in. I do not believe that's a correct way of doing things, and since this was done in error, I'd like to add you back as maintainer for PAPI. That is revert to the state prior to that patch. Apologies again, and let's fix that error. Best regards, Ole -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#18043): https://lists.fd.io/g/vpp-dev/message/18043 Mute This Topic: https://lists.fd.io/mt/78229638/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Python API modules
Hi Paul, > He/Vratko removed me as one of the maintainers of papi. > https://gerrit.fd.io/r/c/vpp/+/22672. I'm cool with not being a maintainer, > kinda funny that it was stuffed into another changeset. I have been called > out repeatedly for submitting unrelated changes ;) My apologies, something has gone wrong here. I wasn't even aware that you got removed. I have reviewed the history and spoken to Vratko to understand what has happened here. Vratko was editing the MAINTAINERS file and made changes in good faith reflecting what he believed was the current state. Expecting reviewers to comment. I missed that in code review, and the patch went in. I do not believe that's a correct way of doing things, and since this was done in error, I'd like to add you back as maintainer for PAPI. That is revert to the state prior to that patch. Apologies again, and let's fix that error. Best regards, Ole -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#18042): https://lists.fd.io/g/vpp-dev/message/18042 Mute This Topic: https://lists.fd.io/mt/78229638/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Python API modules
Hi Paul, Picking off one issue at the time. > Why won't Ole, as maintainer, allow it? > > The build system uses something called vppapigen to generate the c include > files and the json files. It could as easily generate static stubs so that > development with python could be usable without a running vpp instance. I certainly didn't intend for my opinion to come off as a "don't allow". Initially the Python code was fully generated from vppapigen. With a class per .api file and fully formed methods. Later it evolved to dynamically create all functions based on the API representations in JSON. (This latter change made the Python binding independent of VPP version which is how it was possible to individually package it and put it on PyPI, but lets do that on a separate thread.) Generating Python code from vppapigen is simple. But just how do you want these stubs to look like and how should they be packaged and used? If I have pushed back on this it's likely more out of ignorance of the problem rather than ill will. ;-) Best regards, Ole -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#18032): https://lists.fd.io/g/vpp-dev/message/18032 Mute This Topic: https://lists.fd.io/mt/78229638/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Python API modules
Hi Paul ,Thank you for your answear. Iam sure that anyone who uses or maintain this project is very competent, and as a beginner I'm grateful for that and hopefully I Will be able to help it get even biggerWell, my goal is just build a dashboard to make easier the process of managing Vpp, maybe you can help me out with that. If you are interested email me and tell me the best way to keep in touch Best regards Original message From: Paul Vinciguerra Date: 11/13/20 17:29 (GMT-03:00) To: "Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco)" Cc: Marcos - Mgiga , vpp-dev , Ole Troan Subject: Re: [vpp-dev] Python API modules Hi Marcos.Yes. Your assumption is correct. A library user expects access to the library.I'm going to ask the question that you haven't.Why won't Ole, as maintainer, allow it?The build system uses something called vppapigen to generate the c include files and the json files. It could as easily generate static stubs so that development with python could be usable without a running vpp instance.The thing is that there are other trivial issues that need to be addressed first. Wouldn't you love to be able to do 'pip install vpp_papi' and get going?Yes, vpp builds artifacts for vpp_api_python, but who actually wants vpp_api installed globally? The python best practice is to install a venv, and do pip install.Even 'make test' follows this pattern.vpp_papi has changed significantly, yet the version number is never, ever increased.To address this, on April 5, 2019, I attempted to automate versioning of vpp_papi to the git tag. https://gerrit.fd.io/r/c/vpp/+/18694.There is a version on pypi, it is woefully out of date, and honestly not usable. On June 18, 2019, I contributed https://gerrit.fd.io/r/c/vpp/+/20035, to automate the process of updating pypi, with a simple command 'tox -epypi' see:src/vpp-api/python/tox.iniI have also floated the idea of moving vpp_papi to a submodule, so that it could be easily developed against if pypi were too burdensome to add to the release process. Just include the submodule in your code, pin it to any commit you like, and you're off to the races!To be 100% clear, this is not an attack on Ole. I have nothing but respect for him. He is extremely talented and has been *extremely* generous with his time to me. As for me, as anyone here can tell you, I'm not a C programmer. The only reason, I ever considered touching the c code, was because of Ole's help and guidance and his suggestion to fix the API instead of conforming the test. He/Vratko removed me as one of the maintainers of papi. https://gerrit.fd.io/r/c/vpp/+/22672. I'm cool with not being a maintainer, kinda funny that it was stuffed into another changeset. I have been called out repeatedly for submitting unrelated changes ;)All my changesets are still out there, so that should others become blocked, they can still get work done. 'git-review -X 12345' is your friend. (If you use -X instead of -x, you can remove the changeset with 'git rebase -i' if you want to submit a contribution)In that spirit, if you want a python module with static methods, let me know.Paul On Fri, Nov 13, 2020 at 10:57 AM Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) wrote: > All available api are represented in python. That is one part of it. Not every "available" message is useable, as some messages need corresponding VPP plugins, which may be disabled. >> all available Python modules to use in VPP API. It depends on what do you mean by "Python module". If you mean "usable messages", you can examine the message table [0] after PAPI has connected to a running VPP. Vratko. [0] https://github.com/FDio/vpp/blob/66d10589f412d11841c4c8adc0a498b5527e88cb/src/vpp-api/python/vpp_papi/vpp_papi.py#L834-L836 From: vpp-dev@lists.fd.io On Behalf Of Ole Troan Sent: Friday, 2020-November-13 15:51 To: Marcos - Mgiga Cc: vpp-dev Subject: Re: [vpp-dev] Python API modules Hi Marcos, On 13 Nov 2020, at 15:08, Marcos - Mgiga wrote: Hello There, I believe this is a trivial question, but where / how can I get a list of all avaialble Python modules to use in VPP API. You can’t. They are auto generated from the available json representations of .api files. All available api are represented in python. Cheers Ole -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#18025): https://lists.fd.io/g/vpp-dev/message/18025 Mute This Topic: https://lists.fd.io/mt/78229638/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Python API modules
Hi Marcos. Yes. Your assumption is correct. A library user expects access to the library. I'm going to ask the question that you haven't. Why won't Ole, as maintainer, allow it? The build system uses something called vppapigen to generate the c include files and the json files. It could as easily generate static stubs so that development with python could be usable without a running vpp instance. The thing is that there are other trivial issues that need to be addressed first. Wouldn't you love to be able to do 'pip install vpp_papi' and get going? Yes, vpp builds artifacts for vpp_api_python, but who actually wants vpp_api installed globally? The python best practice is to install a venv, and do pip install. Even 'make test' follows this pattern. vpp_papi has changed significantly, yet the version number is never, ever increased. To address this, on April 5, 2019, I attempted to automate versioning of vpp_papi to the git tag. https://gerrit.fd.io/r/c/vpp/+/18694. There is a version on pypi, it is woefully out of date, and honestly not usable. On June 18, 2019, I contributed https://gerrit.fd.io/r/c/vpp/+/20035, to automate the process of updating pypi, with a simple command 'tox -epypi' see:src/vpp-api/python/tox.ini I have also floated the idea of moving vpp_papi to a submodule, so that it could be easily developed against if pypi were too burdensome to add to the release process. Just include the submodule in your code, pin it to any commit you like, and you're off to the races! To be 100% clear, this is not an attack on Ole. I have nothing but respect for him. He is extremely talented and has been *extremely* generous with his time to me. As for me, as anyone here can tell you, I'm not a C programmer. The only reason, I ever considered touching the c code, was because of Ole's help and guidance and his suggestion to fix the API instead of conforming the test. He/Vratko removed me as one of the maintainers of papi. https://gerrit.fd.io/r/c/vpp/+/22672. I'm cool with not being a maintainer, kinda funny that it was stuffed into another changeset. I have been called out repeatedly for submitting unrelated changes ;) All my changesets are still out there, so that should others become blocked, they can still get work done. 'git-review -X 12345' is your friend. (If you use -X instead of -x, you can remove the changeset with 'git rebase -i' if you want to submit a contribution) In that spirit, if you want a python module with static methods, let me know. Paul On Fri, Nov 13, 2020 at 10:57 AM Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) wrote: > > All available api are represented in python. > > > > That is one part of it. > > Not every "available" message is useable, > > as some messages need corresponding VPP plugins, > > which may be disabled. > > > > >> all available Python modules to use in VPP API. > > > > It depends on what do you mean by "Python module". > > If you mean "usable messages", > > you can examine the message table [0] > > after PAPI has connected to a running VPP. > > > > Vratko. > > > > [0] > https://github.com/FDio/vpp/blob/66d10589f412d11841c4c8adc0a498b5527e88cb/src/vpp-api/python/vpp_papi/vpp_papi.py#L834-L836 > > > > *From:* vpp-dev@lists.fd.io *On Behalf Of *Ole Troan > *Sent:* Friday, 2020-November-13 15:51 > *To:* Marcos - Mgiga > *Cc:* vpp-dev > *Subject:* Re: [vpp-dev] Python API modules > > > > Hi Marcos, > > > > On 13 Nov 2020, at 15:08, Marcos - Mgiga wrote: > > > > > > > > Hello There, > > > > I believe this is a trivial question, but where / how can I get a list of > all avaialble Python modules to use in VPP API. > > > > > > You can’t. > > They are auto generated from the available json representations of .api > files. > > All available api are represented in python. > > > > Cheers > > Ole > > > > > > > > > > > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#18024): https://lists.fd.io/g/vpp-dev/message/18024 Mute This Topic: https://lists.fd.io/mt/78229638/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Python API modules
> All available api are represented in python. That is one part of it. Not every "available" message is useable, as some messages need corresponding VPP plugins, which may be disabled. >> all available Python modules to use in VPP API. It depends on what do you mean by "Python module". If you mean "usable messages", you can examine the message table [0] after PAPI has connected to a running VPP. Vratko. [0] https://github.com/FDio/vpp/blob/66d10589f412d11841c4c8adc0a498b5527e88cb/src/vpp-api/python/vpp_papi/vpp_papi.py#L834-L836 From: vpp-dev@lists.fd.io On Behalf Of Ole Troan Sent: Friday, 2020-November-13 15:51 To: Marcos - Mgiga Cc: vpp-dev Subject: Re: [vpp-dev] Python API modules Hi Marcos, On 13 Nov 2020, at 15:08, Marcos - Mgiga mailto:mar...@mgiga.com.br>> wrote: Hello There, I believe this is a trivial question, but where / how can I get a list of all avaialble Python modules to use in VPP API. You can’t. They are auto generated from the available json representations of .api files. All available api are represented in python. Cheers Ole -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#18014): https://lists.fd.io/g/vpp-dev/message/18014 Mute This Topic: https://lists.fd.io/mt/78229638/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Python API modules
Hello There, I believe this is a trivial question, but where / how can I get a list of all avaialble Python modules to use in VPP API. Best Regards -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#18010): https://lists.fd.io/g/vpp-dev/message/18010 Mute This Topic: https://lists.fd.io/mt/78229638/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
