Hi,
Intended audience:
- Pharos contributors;
- installer teams;
- lab owners;
There are some APs summarized at the end, if you want to skip to that.
I propose a few changes trying to achieve the following goals:
- prepare for introducing a strict schema validation for PDFs;
- clean up PDF spec deviations, typos etc. in existing PDFs;
- propose PDF schema based on spec, as strict as possible;
- add tooling for a verify job (to be added later) that checks new PDFs
against the schema;
- propose IDF schema based on current implementations (not so strict);
I apologize for the numerous mails you'll receive to review all these changes.
I am aiming at a clean git history (without merges) for the below topics,
so they'll be easier to track later. This means a lot of rebase and request for
review mails for the Pharos committers.
While working on this, I found a few errors that justify using a schema to
validate PDFs instead of relying on peer review only, e.g.:
- out-of-range/not-in-enum errors;
- type errors;
- wrong features separators;
- missing mandatory properties (unless my schema understanding is wrong);
- wrong key names (typos);
- wrong MAC (I left this one unchanged, because although 'G' is clearly an
invalid char, I'm not sure it should be an 'F' there);
The main topics, their indended audience and goals are summarized below:
1. yamllint-fix: Pharos committers, installer teams;
- skip trying to `eyaml` PDFs that are not encrypted, leading to cleaner
verify (check-jinja) logs, as well as ~25% faster verify jobs;
- skip trying to validate the <config/pdf/pod1*yaml> files, they are not
valid PDFs;
- fix yamllint for output YAMLs generated based on PDF + installer adapter
templates;
- replace encrypted string with (shorter) dummy values during yamllint to
bypass false-positive line-too-long yamllint warnings;
2. check-jinja-cleanup: Pharos committers;
- since we now have a dedicated yamllint CI job for checking PDF files,
let's drop that same check from `check-jinja` and slightly refactor the
output log to make it a bit easier to read;
3. move-net_config-idf: Pharos committers, installer teams, lab owners;
- net_config is a leftover from early IDF development, it wrongfully got
into a few PDFs, althought its place is in the IDF; let's move it there
and deal with the other related properties (e.g. 'fixed_ips') so all
PDFs are compliant with the PDF specification;
- the first step adds support for IDF net_config where missing (Fuel, Daisy);
- next, we move net_config from PDF to IDF for all PODs in all labs;
- the last step removes PDF net_config from installer adapters (Fuel, Daisy);
- NOTE for installer teams: additional changes may be required in the
installer
if additional templates rely on net_config being in the PDF (true for
Fuel).
Ideally, we'd add support IDF net_config in all installers (where required,
this might only be the case for Fuel?), merge the Pharos changes, then
remove
old support for PDF net_config from installers;
4. fix-pdf-spec-deviations: Pharos committers, lab owners;
- minor changes in unused/default values (e.g. disk_rotation for SSD drives);
- add interfaces "name: 'nicN'" where missing;
- fix some typos;
5. pdf-schema: Pharos committers;
- add PDF schema based on the spec (I hope my understanding of the spec is
correct);
- add `check-schema` similar to `check-jinja`, to be leveraged by CI verify;
6. pdf-spec-update: Pharos committers, installer teams, lab owners;
- add new enum value '0' to 'disk_rotation' for ssd/nvme drives;
- use 'name: nicN' for jumpserver interfaces to align with cluster nodes;
- add new enum values 'scsi', 'iscsi' to 'disk_interface' - I am not sure
this is correct;
- I still have some question about certain enums in PDF, but I'll try to
bring those up during the infra meeting on 22nd;
7. idf-schema: Pharos committers, installer teams;
- add schema for IDF: strict for Fuel section, not so strict for Daisy or
the net_config section yet;
8. config-license: Pharos committers
- update copyright year in files we touched;
- add license headers where missing;
- this is separate from all of the above because we might want to refactor
licensing by adding a single LICENSE file in each dir;
Proposed action points:
1. Pharos committers
- please scan all of the above, input is greatly appreciated;
2. Installer teams
- most likely, no changes are expected in the installers' code (except for
Fuel, which I'll handle myself);
- a quick scan of the above is recommened for Daisy & Fuel contributors;
3. Lab owners
- [all] all labs are affected, changes to PDFs are split by lab name;
most of them are simple typo fixes and don't require input;
- [Ericsson, Intel, BII] there are some missing user/pass fields, most likely
removed during the move from securedlab to pharos git repo.
This should be handled in one of two ways:
* add cleartext IPMI user/pass, if you don't consider this
security-sensitive;
* add encrypted IPMI user/pass, as described in [9];
BR,
Alex
[1] https://gerrit.opnfv.org/gerrit/#/q/project:pharos+topic:yamllint-fix
[2] https://gerrit.opnfv.org/gerrit/#/q/project:pharos+topic:check-jinja-cleanup
[3] https://gerrit.opnfv.org/gerrit/#/q/project:pharos+topic:move-net_config-idf
[4]
https://gerrit.opnfv.org/gerrit/#/q/project:pharos+topic:fix-pdf-spec-deviations
[5] https://gerrit.opnfv.org/gerrit/#/q/project:pharos+topic:pdf-schema
[6] https://gerrit.opnfv.org/gerrit/#/q/project:pharos+topic:pdf-spec-update
[7] https://gerrit.opnfv.org/gerrit/#/q/project:pharos+topic:idf-schema
[8] https://gerrit.opnfv.org/gerrit/#/q/project:pharos+topic:config-license
[9] https://github.com/opnfv/pharos/blob/master/config/utils/README.eyaml.rst
_______________________________________________
opnfv-tech-discuss mailing list
[email protected]
https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
