Hi All,
After doing METRON-744, where I had to walk through a manual test of every
place that Stellar touched, it occurred to me that we should script this.
It also occurred to me that some scripts that are run by the PR author to
ensure no regressions and, eventually maybe, even run on an INFRA instance
of Jenkins would give all of us some peace of mind.
I am certain that this, along with a couple other manual tests from other
PRs, could form the basis of a really great regression acceptance-test
suite and I'd like to propose that we do that, as a community.
What I'd like to see from such a suite has the following characteristics:
- Can be run on any Metron cluster, including but not limited to
- Vagrant
- AWS
- An existing deployment
- Can be *deployed* from ansible, but must be able to be deployed
manually
- With instructions in the readme
- Tests should be idempotent and independent
- Tear down what you set up
I think between the Stellar REPL and the fundamental scriptability of the
Hadoop services, we can accomplish these tests with a combination of shell
scripts and python.
I propose we break this into the following parts:
- Acceptance Testing Framework with a small smoketest
- Baseline Metron Test
- Send squid data through the squid topology
- Add an threat triage alert
- Ensure it gets through to the other side with alerts preserved
- + Enrichment
- Add an enrichment in the enrichment pipeline to the above
- + Profiler
- Add a profile with a tick of 1 minute to count per destination
address
- Base PCap test
- Something like the manual test for METRON-743 (
https://github.com/apache/incubator-metron/pull/467#issue-210285324)
Thoughts?
Best,
Casey
