GitHub user ottobackwards opened a pull request:
https://github.com/apache/metron/pull/856
METRON-1339 Stellar Shell functionality to verify stored stellar statements
This will allow users to check their deployed statements, say after
upgrade, when they are at rest ( and would fail on use ).
In other words, they were valid when stored, but are not now because of
stellar changes, such as new keywords.
The interface `StellarConfiguredStatementReporter`, which is
`@IndexSubclasses` ( ClassIndex) marked, allows the shell to discover reporters
that can provide statements for validation. This discovery allows de-coupling
of stellar and 'hosts' that know about the location of the stored statements,
and the configuration structure details.
> We do mention the configurations in the shell output at this time.
`metron-common` implements this interface, and can run through visiting all
the configurations.
A new magic keyword was added ` %validate_configured_expressions`
When executed, the shell
- discovers the reporters through class index
- visits the reports, with callbacks for visits or errors
- per visit ( which is called for a specific stellar statement ) the
statement is compiled and errors reported
- if the entire config fails ( threat triage stellar errors fail on
deserialize so we don't get to do ANY enrichment visits in that case ) the
error callback handles that
I'm getting this out there, still a couple of things todo:
1. full dev run. I have been testing with stellar external to full dev
iteratively
2. readme
3. steps to test
4. unit test
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to
be created at [Metron
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON-XXXX where XXXX is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically master)?
### For code changes:
- [ ] Have you included steps to reproduce the behavior or problem that is
being changed or addressed?
- [ ] Have you included steps or a guide to how the change may be verified
and tested manually?
- [x] Have you ensured that the full suite of tests and checks have been
executed in the root metron folder via:
```
mvn -q clean integration-test install && build_utils/verify_licenses.sh
```
- [ ] Have you written or updated unit tests and or integration tests to
verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] Have you verified the basic functionality of the build by building
and running locally with Vagrant full-dev environment or the equivalent?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in
which it is rendered by building and verifying the site-book? If not then run
the following commands and the verify changes via
`site-book/target/site/index.html`:
```
cd site-book
mvn site
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ottobackwards/metron
stellar_verify_deployed_shell
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metron/pull/856.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #856
----
commit a5087f3a170eeda6ee778397c919d9eddd5597e2
Author: Otto Fowler <ottobackwa...@gmail.com>
Date: 2017-11-30T14:15:40Z
Stellar shell functionality to verify stellar statements.
This will allow users to check their deployed statements, say after
upgrade, when they are at rest ( and would fail on use ).
In other words, they were valid when stored, but are not now because of
stellar changes, such as new keywords.
The interface StellarConfiguredStatementReporter, which is @IndexSubclasses
marked, allows the shell to discover
reporters that can provide statements for validation. This discovery
allows de-coupling of stellar and 'hosts' that
know about the location of the stored statements, and the configuration
structure details.
We do mention the configurations in the shell output at this time.
metron-common implements this interface, and can run through visiting all
the configurations.
----
---
