Hi, See inline. BR, Alex
From: david.blaisonn...@orange.com [mailto:david.blaisonn...@orange.com] Sent: Tuesday, September 26, 2017 11:54 AM To: Alexandru Avadanii; opnfv-tech-discuss@lists.opnfv.org Cc: jack.mor...@intel.com; Guillermo Herrero Subject: Re: PDF feedback and questions from our experience with Fuel Hi, I hope I don't offend you about net_config, it is an hard subject, not easy to summarize, and born during summer holidays without everyone around the gerrit pit. [ Alex ] Don’t worry about offending anyone, we are here to create a good design, not to enforce one or the other’s preferences. About network naming, I understand the need of anonymize the networks parts to not over specify the installer level, but by calling them 'networkX' we will lost a precious information: 'how this network is prepared' by ops in network and security aspects. You may had already talk about that yesterday (sorry for missing it, job and child schedules collision), but simple networking questions must be answered when preparing a network mapping: Is this network NATed to outside ? does this network can talk to one other ? are some ip/ports opened at firewall level ? In Orange side, and for security aspects, we isolate some networks: Public can not talk to admin, storage is isolated and is not NATed to outside... Using 'admin', 'storage', 'public' naming give a part of the information, but 'networkX' don't, except if we are adding a description in the PDF like flow matrix, or we add external source, like the wiki. How do you think to handle that part ? [ Alex ] The argument for renaming the networks was that the PDF should only describe the hardware setup, leaving out any software aspects, like the usage of those networks by each installer. This, of course, makes things like describing NAT / isolation harder, but then again, I don’t think those belong in the PDF either. I have no strong opinion towards ‚admin’ vs ‚vlan0’ naming, the Fuel adapter can work very well with any implementation. I also proposed we let the PDF maintainer choose the network names, but this idea was rejected since we want to keep the PDF strict. If others have any input on this, I would invite them to propose a naming that solves as many of the problems above as possible, or at least defend one solution in favor of the other. This raise a more general question, is there recommendations about network topology and security rules in OPNFV pods or each lab is follow its own rules ? For C section, here are my answers: 1. 'vlan: 0' vs 'vlan: native': OK for native if 0 have a special meaning 2. 'disk_rotation: ssd' or 'disk_rotation: 15000' for SSD drives. Is disk_rotation enough to specify a disk performances? don't we also need the size of the cache ? we would better talk about bandwidth if we want also use it for ssd ? Maybe Bottlenecks project can help us. 3. 'features: dpdk|sriov' vs 'features: dpdk, sriov': it seems to be a collision between PDF specs and the yaml one. If it is a list we should simplify the installer parsing by using a yaml list: [ x, y, z] or a dashed list 4. 'features: null' vs 'features: ' vs omitting it altogether: 'features: []' or omitting; the first is yaml compliant with a list, the second is to simplify the pdf. 5. inconsistent node naming: in my opinion, this part shall be reserved for the ops, it is the only way for them to map a node with a physical server in the DC (except the ipmi address that is not the simple way to find a node). All values shall be authorized, but recommandation can be made on 'pod<x>-<function>') 6. Address/netmask: isn't netmask information in address overlaping with net_config netmasks ? 7. 'os' in POD nodes: Only in jumphost 8. IPMI interface MAC on the 'interfaces' list: no, it overlaps with remote_management/mac_address and is not seen from the os. Thanks again for this huge work of listing all pending points. BR David -- David BLAISONNEAU NFV platforms DevOPS - OPNFV contributor Orange - IMT / OLN / CNC / NCA / SINA tel: +33 (0)2 96 07 27 93 email: david.blaisonn...@orange.com<mailto:david.blaisonn...@orange.com> 2, avenue Pierre Marzin 22307 LANNION Cedex - France Le 25/09/2017 à 20:02, Alexandru Avadanii a écrit : Hi, The part about net_config was confusing because it was not (yet) part of the official PDF spec. After today’s infra meeting, and some more discussion on IRC, it looks like net_config will become part of the spec, but we need to sort out some network naming issues first. However, ‚admin’, ‚mgmt’ & co network names are not inline with the PDF idea of describing strictly the hardware, so they will most likely need to be mapped using the installer descriptor file companion for now. I updated [9] to reflect this decision. It makes things a little bit more complicated for the installer adapters, but I think keeping PDF clean (with no higher-level specifics) is a good idea overall. Sections C and D in my first mail still need to be sorted out, but they are mostly minor quirks or deviations from the spec, so they are not blocking the release of PDF. BR, Alex From: david.blaisonn...@orange.com<mailto:david.blaisonn...@orange.com> [mailto:david.blaisonn...@orange.com] Sent: Monday, September 25, 2017 12:11 PM To: Alexandru Avadanii; opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org> Cc: jack.mor...@intel.com<mailto:jack.mor...@intel.com>; Guillermo Herrero Subject: Re: PDF feedback and questions from our experience with Fuel Hi Alexandru, thanks for this big set of patches and a sum up of our last pdf discussions. The part about net_config is quite confuse, specially because you sum up many problems already solved by net_config, but talking about PDF not having it, then talking about what add net_config... but it is a hard job to summerize those evolution and point that we would better understand each one by using PDF version id, and work on a reference model. Generally it will be +1 :) BR David -- David BLAISONNEAU NFV platforms DevOPS - OPNFV contributor Orange - IMT / OLN / CNC / NCA / SINA tel: +33 (0)2 96 07 27 93 email: david.blaisonn...@orange.com<mailto:david.blaisonn...@orange.com> 2, avenue Pierre Marzin 22307 LANNION Cedex - France Le 25/09/2017 à 00:21, Alexandru Avadanii a écrit : Hi, A. Current Fuel PDF integration status Fuel and Armband teams have been working on moving to the new PDF as a single input configuration file. We have proposed a new installer adapter template for Fuel in Pharos [1], as well as new PDFs in securedlab for the PODs Fuel uses: - lf-pod2 [2]; - arm-pod5 has been around for a while; - ericsson-pod1 and zte-pod1 already have PDFs, but might require smalls updates to work with the Fuel installer adapter; While working on the PDFs, we proposed some patches that should improve/extend the verify job for securedlab, use the Pharos git repo for PDF parsing, respectively some minor cleanup/code folding: - securedlab verify job should switch to using Pharos installer adapters and generate_config [3]; - yamllint fixes and code folding for existing PDFs [4]; - add verify job summary, run the whole test matrix instead of bailing on the first error [5]; - extend verify with yamllint runs for PDF files, as well as output yaml file(s) generated by check_jinja2 [6]; - fix missing IPMI credentials for lf-pod4 (caught by linting the output yaml described above) [7]; - (unrelated to PDF, Fuel cleanup) remove old Fuel configuration files we no longer use [8]; B. PDF specification limitations for Fuel Tbh, I have a hard time summarizing this, but I'll try. Currently, we have some PDFs that define a 'net_config' section (global per PDF), while the spec and most PDFs don't. This resulted from: - the need to support multiple VLANs over the same physical interface; - installers expecting a network-centric mapping between existing/to-be-created networks and available interfaces; Looking at what Fuel expects as input, we'd like to be able to easily map IPs in a certain network (e.g. admin, mgmt, private, public) to a particular interface name. But the PDF does not allow specifying interface names directly, as those depend not only on target OS, but also on OS config (e.g. net.ifnames=0 biosdevname=1 vs net.ifnames=1 biosdevname=0). Indexing interfaces is also fragile, as a bios upgrade might change PCI layout and therefore the NIC ordering (quite rare, but still). What happens if we add a new interface that ends up at index 1, between existing interfaces 0 and 2? The 'net_config' section is a compromise that solves part of the problem, but still does not provide a solid solution for the physical interface name mapping, as it also relies on interface index. What if the controller nodes have a different set of interfaces than the compute nodes have? Adding 'net_config' to PODs aligned with the current spec is also problematic, as it'd duplicate the VLAN information, making the whole thing even more fragile. Tl;dr I submited a proof of concept refactor of net_config, which triesto align more with the current PDF spec, although it is not 100% compatible (we can't model multiple VLANs on the same physical interface with the current spec) [9]. Observations wrt this change: - network-centric approach, installer-friendly; - physical interface to virtual interface mapping is NOT 1:1 (multiple VLANs on same physical NIC); - virtual inteface to network mapping is 1:1; - network to IP mapping is 1:1; - networks are global for the POD (including network to virtual network); - network to physical inteface mapping is NOT global per POD, and should be overrideable on a per-node basis; - features and speed params were silently discarded during net_config addition, bring them back for the physical intefaces; - converting from current spec-compatible PDF to this proposed format should be trivial for PDF files, and should require very little work on the installer adapters; Etc. Please take some time to review this and let's come up with a better solution if we can find one, or at least align our current PDFs to one format or another. The PoC does not solve the interface indexing issue, but at least it provides a mechanism that makes it per-node configurable. C. PDF current implementation issues This section covers some divergent aspects in the current PDFs in securedlab. The PDF spec is quite clear on some of these, so I don't understand why there are so many mutations in the wild. If parsing the PDF is hard / does not align with the installer-expected input, we can define some new custom filters in generate_config.py, although most of these should be fixable within the current tool set. 1. 'vlan: 0' vs 'vlan: native' The spec uses 'native' to differentiate from '0', which might have a special/reserved meaning on some network equipments. We currently have both formats in securedlab. 2. 'disk_rotation: ssd' or 'disk_rotation: 15000' for SSD drives Here, the spec does not provide a special value for SSDs. Afaik, no installer adapter consumes this yet, but we should extend the PDF spec and then adhere to the new value. We currently have a mix of both formats in securedlab, and both are wrong imo :). 3. 'features: dpdk|sriov' vs 'features: dpdk, sriov' Again, the spec is quite clear on the format using '|', but we have divergent implementations in the wild. 4. 'features: null' vs 'features: ' vs omitting it altogether We should agree on the preffered value for no features and align all PDFs. 5. inconsistent node naming Some PDFs use 'pod1-jump' + 'pod1-node1', while others go for different, custom names (e.g. 'lfpod4-jumpserver' + 'lfpod4-node1', 'CI-POD1-HOST' + 'CI-ERICSSON-POD1-NODE1'). Not the biggest issue, but it'd be nice to agree on a common format here. 6. Address/netmask The spec defines all adresses as IP/mask, e.g. 10.20.0.2/24. However, this is hard to parse by different installers, so in practive we see multiple occurences of specifying just the IP address without the mask. Would it be possible to split the spec format in 'address: 10.20.0.2' + 'netmask: 24' (or 'netmask: 255.255.255.0')? If not, we should agree on the format going forward, and update all current occurances that don't respect it. Note that this also applies to IP address in 'remote_management'. 7. 'os' in POD nodes Imo, this parameter does not belong in the PDF at all - it makes sense for the jump server, since that is preinstalled, but not for the other nodes. I think it should be removed, especially since some PDFs only define it for the first node and not for the rest ... 8. IPMI interface MAC on the 'interfaces' list Should the IPMI interface be present in the 'interfaces' list? I think I saw this in 1 or 2 PDFs, so I thought I should ask here first, before removing it. D. Installer adapter issues 1. Most installer adapter templates generate invalid YAML files (only JOID and the proposed Fuel pass this test, and not for all PODs); 2. Some installer adapter templates rely on the global 'remote_params', which is not mandatory, plus each node might define its own parameters; e.g.: ipmi_user: {{ conf['jumphost']['remote_params']['user'] }} # wrong, relies on non-mandatory 'remote_params' (which had a different name in pharos/config/pod1.yaml for a start), does not allow per-node creds ipmi_user: {{ conf['nodes'][0]['remote_management']['user'] }} # right etc. There are multiple minor design issues in the current installer adapters, and I'm sure each installer team knows about how fragile the current templates are. I saw some comments in the templates as well, so I'm sure I don't have to re-iterate that here. The above is not an exhaustive list, it merely covers the questions I gathered while working with PDF + Fuel. I didn't proofread this, sorry for any mistakes that slipped through. See you at today's Infra meeting. BR, Alex [1] https://gerrit.opnfv.org/gerrit/#/c/42759/<https://url10.mailanyone.net/v1/?m=1dwPQM-0000Xa-54&i=57e1b682&c=8qHxZYlWQXu5hq1IvOG5wgJ3uMDQMPvkr42VBre0mAWYNww5GFXnxeyR7a_0y1YJrb0zFj2Mifv3p7EYXiGn0TKXdXauroHCEfK21EZwn5j-fe67ApoHlB7SadKouuXG7Qhj2RyO85nupTquG2KXo-PhqiMkq3iczARua3QIKZYLlgn0inZZVfNCAhgalj26npK7TzM3deerBBkjuYPROQe34_UlIz8tia3cZvq_VfSFa66XrCpHbrw63leJOpT2> (Add fuel installer adapter) [2] https://gerrit.opnfv.org/gerrit/#/c/42875/<https://url10.mailanyone.net/v1/?m=1dwPQM-0000Xa-54&i=57e1b682&c=6W5MHW0Yl58dBvnmN2kykZ8E-9b08eVKp-prilC-R_nXzA-aNKXdoIWjblAWZcAQ9co2H2qXm-Agc5O_V6WCX2QPGh3wevmI77QRrQ1LRZ46qk6ZG_ygFao1RUVxVd-kUzJCAXrURU0yrjQoASU7QPxi4hQongww8EBllkv5-edfnwSLmia37dIImF7uqAosVYWiPpPa6qfs2KeIqg9d13SiXqZ2rj1IX5T_vMzGBG3acU-gMkFi-5DDyq2XywaF> (lf-pod2: Pod Descriptor File) [3] https://gerrit.opnfv.org/gerrit/#/c/42343/<https://url10.mailanyone.net/v1/?m=1dwPQM-0000Xa-54&i=57e1b682&c=w60KwSAGi5DyDS_zSDaHI81Do-BYLeEnniqW0Euz0i4Nhd3njClS35hPhB-ikANxPlTNxkInFwnF8LqVI9zOPDsXOoLKiCKNU6w5_-OSB5NiEAgLbgo-mAThDJFRVCyTw417m8cQ-XnXYsDU7KVl9Y3mx-5ptHrxjWXjgA_6Uzs1GE8oDfUGkD5LiJUbS3sE-OqWY3EO5diO68D_dVDZzgdYvRJ4jGVn_Ax6a1we-gLWj3mrNM7v2yeNHnp5AxdD> (securedlab: Use Pharos git sub for PDF validation) [4] https://gerrit.opnfv.org/gerrit/#/c/42729/<https://url10.mailanyone.net/v1/?m=1dwPQM-0000Xa-54&i=57e1b682&c=CW9RBVdhMgRN3jPaWshFo7B9cWA7O1AvdEM4jsilFMmy2rw5U8dRdTyIk545-Y104YXdNMljhgXdVzugYXSzGkAm7sPQOfjBs3q09uKsJpmH18GOxWijCQYRlLIasGlvbV-cBZw5cn4aRD9jRmxguNHA54rps_NY5v8Z6N5k3mWfW30_eMYJKrYis7hYDYPtRMPjob8_fCOhcnihxTARIbzHughWBzT1DR69fvELyoYdPjMtez59BTL7aVfDnfAr> (PDF: Fix yamllint warnings & fold reusable code) [5] https://gerrit.opnfv.org/gerrit/#/c/42599/<https://url10.mailanyone.net/v1/?m=1dwPQM-0000Xa-54&i=57e1b682&c=B3o1LJhCOcPEb9QKWMMUfkd_YXAsX7CqNS4ur4awLhWxE0yVJCmuc9zPaptw6a6YxjnooQ11e_G5-BK5QRCOX0TVY5jJqYoReViobptxDW8ZQh0wFEx3KuQx5LQv_UaXqH0j5WWzsD5VdUMZe8KD3s8I3YBXbe_IM7fil0rAqNkFSYYHxO035xNfBiB4cEQOumT-pSwXKUK0A4jeV1NtCgsgTH8vlRNdryTbxyMvv8jiiqgV8rUPUT_21JrLRdCX> (PDF: Add result summary to check-jinja2) [6] https://gerrit.opnfv.org/gerrit/#/c/42711/<https://url10.mailanyone.net/v1/?m=1dwPQM-0000Xa-54&i=57e1b682&c=7zdLZo1ktWzhK7o2M39clwfZo_uXdlt_p34HqiFrQxYEGYX-aeySDmGU1ff4flvwx2dyTsuEDqE8lCe8NZjyyluQwfK9TvUCLakEIlfMgRqL_u0H4zeHrprIlrNYItHXTNno79us7Sg_D0H_RN8VQuCekoaX26TP5Y-6sNRYwaRQACtCd-h3l0MWAddg-khRt7llQfLkOqLD4ZLwXuyAjG3j0TlsIlCznH_1hfITFOiLjNNhO2Oif2lHEwgDY542> (PDF: Run YAML Linter on pod descriptors / output) [7] https://gerrit.opnfv.org/gerrit/#/c/42881/<https://url10.mailanyone.net/v1/?m=1dwPQM-0000Xa-54&i=57e1b682&c=kFRihUb2jxDORQmMAfBSZYo6xalEREk-kJZUH05TscBLfcf_cUgr-1zjtgqo9Nc8Yi9d9Xx8mMjqH9Xfpny1Zu9zh_gHQlw0m2HmHc6pnw2EK6wDycqQtd2FVtnoWlcQpKBytuMZdBiNv0wSxwslmT0sAa8TcTE_b6jf-pm4W3tRCQdMOoN6WijJpzlTudPor8gxsEx-Myz_5GOsoPu06Ut75meoeySWAZNgY6Rbt3oZTUuqTGqtWAwjP2cGcoiM> (lf-pod4: Add missing IPMI user/pass) [8] https://gerrit.opnfv.org/gerrit/#/c/42805/<https://url10.mailanyone.net/v1/?m=1dwPQM-0000Xa-54&i=57e1b682&c=XRvBikeWqpDNB5Ai8Yz4T68CqKRgs9UplwT6gSbzdoTbRI4GjW0NPETVXmsQr92kmGZr0VKHCXOigBTpY_sz32y3KZOlqsqbM9Yz3-3DgKdlEShmkE3IkvkJCuTV5OKbJMOCcegKRgW_gAhfga7HbreRH9syrrPex2mH-dChrAtg8944zKK2nSal2-Nkw4jYXHFoGmUFbVXpcAc_GqxCYtcETGJfmmWOYs4gTq4dpOrcal6uX-u25xvHoMiMWPjZ> (cleanup: fuel: Remove obsolete reap, dea, dha) [9] https://gerrit.opnfv.org/gerrit/#/c/42893/<https://url10.mailanyone.net/v1/?m=1dwPQM-0000Xa-54&i=57e1b682&c=Py0nDvBPq3M2Z-ippJcVXVS5JFGOBQQABqd3aGETWkQVCnXYR6Jg6gf-U-4uB_1PraWLe4wjIKsMjQ0SgOBCi-5nsQSMgly_qbuy3UMtxQhw0g-Ar4Y-pvppUF2GG2uDzMTETtkRswRmj37tP537r7HaT0WC8FGijmA_6t5FSUjNdDdojhLdetReqzd4T6HhZYeFMJeZ1QS3KKDf2rt4fPrVS_K0x8SH72EgLqzJuhh2hKvrW3kkFY8qxERtwsbq> (PoC: net_config refactor) _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you. _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you.
_______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
