mattcasters opened a new pull request, #8637:
URL: https://github.com/apache/hop/pull/8637
Addresses #6744
### Description
This pull request adds capabilities to generate and validate W3C XML schemas
(`.xsd`) for Apache Hop metadata:
1. **Pipeline XML Schema**: `pipeline.xsd` describing pipeline definitions
(`.hpl`).
2. **Workflow XML Schema**: `workflow.xsd` describing workflow definitions
(`.hwf`).
3. **Transform XML Schemas**: Individual XML schemas for all registered
transform plugins in `PluginRegistry` (`transforms/<pluginId>.xsd`).
4. **Action XML Schemas**: Individual XML schemas for all registered action
plugins in `PluginRegistry` (`actions/<pluginId>.xsd`).
### Key Features & Architecture
- **Core Schema Generator (`XmlMetadataSchema`)**:
- Dynamically reflects class fields annotated with `@HopMetadataProperty`.
- Supports Java primitives, numeric types, dates, enums (`IEnumHasCode`),
boolean variations (`HopBoolean`), grouped/ungrouped lists, maps
(`storeMapAsList`), inlined POJOs, interface type substitutions, and
polymorphic dispatch.
- Emits element-order independent schemas (`<xs:choice minOccurs="0"
maxOccurs="unbounded">`) by default to match Hop XML serialization.
- Prevents W3C XSD Unique Particle Attribution (UPA) violations by
emitting open content models for unsubstituted interface fields
(`ITransformMeta`, `IAction`) and exact concrete typed elements for plugin
schemas.
- **Engine Service & Validator**:
- `HopXmlSchemaService`: Coordinates schema generation and batch export
via Apache Commons VFS (`FileObject`, `HopVfs.getOutputStream()`).
- `HopXmlSchemaValidator`: Validates XML documents or VFS files against
generated schemas using JAXP `SchemaFactory`.
- **Command Line Interface (`hop xml-schema`)**:
- CLI command plugin (`@HopCommand(id = "xml-schema")`, alias: `schema`).
- Options matching the GUI dialog: `-t, --target-folder`, `-p,
--pipeline`, `-w, --workflow`, `--transforms`, `--actions`, `-f, --filter`,
`-s, --strict-order`.
- Additional options: `-a, --all` (default export behavior) and `-v,
--validate <file>` to validate a pipeline or workflow against its schema.
- Automatically registers aliases on the root Picocli command in
`Hop.java`.
- **Hop GUI Integration**:
- Main menu action under **Tools -> Generate XML Schemas...**.
- Dialog implemented using annotated metadata model
(`XmlSchemaExportDialogModel`) and `GuiCompositeWidgets.addScrolledComposite`
with progress monitor.
- **Documentation**:
- Added user manual page: `hop-tools/hop-xml-schema.adoc` with dialog
screenshot and CLI guide.
- Updated navigation menus and index pages.
- **Testing**:
- Unit tests in `XmlMetadataSchemaTest` and `HopXmlSchemaServiceTest`.
- Validates generated schemas against real sample files
(`abort-basic.hpl`, `check-db-connections.hwf`).
------------------------
- [x] Run `mvn clean install apache-rat:check` to make sure basic checks
pass.
- [x] If you have a group of commits related to the same change, please
squash your commits into one and force push your branch using `git rebase -i`.
- [x] Mention the appropriate issue in your description (for example:
`addresses #123`), if applicable.
- [x] I hereby declare this contribution to be licensed under the [Apache
License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
