[
https://issues.apache.org/jira/browse/METRON-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16068492#comment-16068492
]
ASF GitHub Bot commented on METRON-947:
---------------------------------------
GitHub user ottobackwards opened a pull request:
https://github.com/apache/metron/pull/626
METRON-947 [DO NOT MERGE UNTIL METRON-942] Stellar Management Function for
Metron Parser Extension Configurations
METRON-777 adds the Metron Extension capability, the default parsers as
extensions, and the parser archetype
METRON-942 adds metron-rest support for installing and uninstalling parser
extensions, as well as the configuration primitives for extensions in zookeeper
This PR adds a Stellar Management command to retrieve those configurations.
## Testing
### Build and install some parser extensions
Follow the test plan for
[METRON-942](https://github.com/apache/metron/pull/580), to the point where you
have built and installed one or more parser extensions.
### deploy the metron-management jar per readme
### Run Stellar
```bash
[vagrant@node1 ~]$ /usr/metron/0.4.0/bin/stellar -z node1:2181
```
Run the command:
```bash
CONFIG_GET('PARSER_EXTENSION', 'metron-parser-nice-assembly-0_4_0')
```
verify that the configuration return matches what you would see in the rest
client
if you executed Parser Extension Controller : get
/api/v1/ext/parsers/{name} -> name = metron-parser-nice-assembly-0_4_0
### For all changes:
- [ x] Is there a JIRA ticket associated with this PR? If not one needs to
be created at [Metron
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [ x] Does your PR title start with METRON-XXXX where XXXX is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [x ] Has your PR been rebased against the latest commit within the target
branch (typically master)?
### For code changes:
- [na ] Have you included steps to reproduce the behavior or problem that
is being changed or addressed?
- [ x] Have you included steps or a guide to how the change may be verified
and tested manually?
- [x ] Have you ensured that the full suite of tests and checks have been
executed in the root incubating-metron folder via:
```
mvn -q clean integration-test install && build_utils/verify_licenses.sh
```
- [x ] Have you written or updated unit tests and or integration tests to
verify your changes?
- [ na] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ x] Have you verified the basic functionality of the build by building
and running locally with Vagrant full-dev environment or the equivalent?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in
which it is rendered by building and verifying the site-book? If not then run
the following commands and the verify changes via
`site-book/target/site/index.html`:
```
cd site-book
mvn site
```
#### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
It is also recommended that [travis-ci](https://travis-ci.org) is set up
for your personal repository such that your branches are built there before
submitting a pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ottobackwards/metron METRON-947
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metron/pull/626.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #626
----
commit 864d320d91c522dfc2eb63fc12341f316a3f8952
Author: Otto Fowler <ottobackwa...@gmail.com>
Date: 2017-03-17T04:56:49Z
Metron Extension system
Based on Apache Nifi Nars
NAR changes
* new lib , rebrand to bundles from NAR
* port to VFS/FileObject from File based
* ability to set property values
* Rework FileUtils so that you can derive and override
* added initializers to set 'classes' that we care about instead of hard
coding them, still needs defaults
* added components nec. for integration tests ( do not want dep. on
metron-* )
* VFSClassloader for NarClassLoader
* Hdfs based integration test version of unpacknars tests
* HDFS ( filesystem ) based fileutilities to cover for writes to hdfs,
since VFS is currently R/O HDFS
* modified plugin to support configuration of outputs
* use class index not service loader ( both subclass and annotated
supported )
Archetype
* Parser Extension archetyp
* incudes all configuration
* creates tar.gz with bundle and configuration
* class index support ( automatic generation )
Extensions
* new extensions modules
* parser
* archetype built module for each parser type
* support for configuration only parsers with tests
Parsers
* moved all but json, csv, grok to extensions
* Bolt now loads from bundle properties
Deployment
* rpms for parsers
* create extension directories
* ambari initializes zookeeper per parser
* amabri creates hdfs directories
* ISSUE: Writing to hdfs
Rest-API
* only test against parsers in metron-parsers
* still needs integration
commit 850154cb086bd9f825daa6797b575cde95392a87
Author: Otto Fowler <ottobackwa...@gmail.com>
Date: 2017-04-14T12:55:37Z
fix package names
commit e8caaf51e3f3d640e9ec1c3c548bd722d595687d
Author: Otto Fowler <ottobackwa...@gmail.com>
Date: 2017-04-17T12:19:02Z
Merge remote-tracking branch 'apache/master' into METRON-777
commit c18cbe3a8847fbed7b5f6ba9182b58f01d142c2c
Author: Otto Fowler <ottobackwa...@gmail.com>
Date: 2017-04-23T16:14:15Z
merge apache/master
commit 7760425b02142f01f37d58876aa09fb88b197c82
Author: Otto Fowler <ottobackwa...@gmail.com>
Date: 2017-04-27T10:45:37Z
Merge remote-tracking branch 'apache/master' into METRON-777
commit bc7cc2d823d54be58fae13ef94f671457d4c10f0
Author: Otto Fowler <ottobackwa...@gmail.com>
Date: 2017-04-29T04:05:41Z
use correct testing bundle
commit c25585d10529ca2048d2ed0df4d8dbd904d3db17
Author: Otto Fowler <ottobackwa...@gmail.com>
Date: 2017-04-29T04:07:11Z
use the configured extension list, do not hardcode
commit 158f463f121cbca2cf2c4d3c1cd9cf7b707e1d23
Author: Otto Fowler <ottobackwa...@gmail.com>
Date: 2017-04-29T12:58:41Z
Merge remote-tracking branch 'apache/master' into METRON-777
commit e21efb89a37d034e36cb82dab6c0b862cad3777b
Author: Otto Fowler <ottobackwa...@gmail.com>
Date: 2017-04-30T13:16:41Z
make dependency explicit, I think parent version is not what we think it is
when running from archetype, causing the extension versio to be used for this
dependency
commit ac92d7b88404f8b8bc825c71a4aeee5aa015d757
Author: Otto Fowler <ottobackwa...@gmail.com>
Date: 2017-04-30T13:44:58Z
instead of hard-coding in metron-parsers-extensions, overload in the
archetype to use metronVersion
commit fd021e42a2cdb80676426b65ae227ac9f44fcd6a
Author: Otto Fowler <ottobackwa...@gmail.com>
Date: 2017-05-03T14:34:31Z
Merge remote-tracking branch 'apache/master' into METRON-777
commit 9cafe970b40d551a846fc442ac121591ab1d0d6d
Author: Otto Fowler <ottobackwa...@gmail.com>
Date: 2017-05-04T02:18:29Z
specify the plugin to fix dependency problem
commit de48845634ae1cecb258175326592a28f5b3f8fe
Author: Otto Fowler <ottobackwa...@gmail.com>
Date: 2017-05-04T02:40:55Z
second attempt to fix plugin errors with jacoco
commit 995a4d746fa21400a55e9571957878ef2d5b48d8
Author: Otto Fowler <ottobackwa...@gmail.com>
Date: 2017-05-04T13:46:58Z
Merge remote-tracking branch 'apache/master' into METRON-777
commit 5f74fc3a15fe5e56f9e3483fa4c25961ffb284dd
Author: Otto Fowler <ottobackwa...@gmail.com>
Date: 2017-05-04T20:25:18Z
be sure to clear out before and after, we cannot assume order of maven build
commit a3e63555e4e76486be58ddd6f818006ff42cdd1f
Author: Otto Fowler <ottobackwa...@gmail.com>
Date: 2017-05-05T03:42:54Z
Merge remote-tracking branch 'apache/master' into METRON-777
commit 1e0e30562b4b526d9512bb1dc06077674d6b3277
Author: Otto Fowler <ottobackwa...@gmail.com>
Date: 2017-05-05T18:18:32Z
update archetype based on changes to loading and configuration
commit e3b71aed74271cf255be5fad072a99506f553d2e
Author: Otto Fowler <ottobackwa...@gmail.com>
Date: 2017-05-06T16:08:58Z
use simple json and not the JSONUtil Instance, there are issues in storm
loading
commit 6c0201b853ffd079f02589011c57cb6a960475e8
Author: Otto Fowler <ottobackwa...@gmail.com>
Date: 2017-05-09T13:55:31Z
Merge remote-tracking branch 'apache/master' into METRON-777
commit ab0aad849319117dd3d964247861cf09bc3e7822
Author: Otto Fowler <ottobackwa...@gmail.com>
Date: 2017-05-10T12:57:21Z
Merge remote-tracking branch 'apache/master' into METRON-777
commit 5565ad661f4143234b67230f7cfd420ea1bce91a
Author: Otto Fowler <ottobackwa...@gmail.com>
Date: 2017-05-10T19:47:58Z
Merge remote-tracking branch 'apache/master' into METRON-777
commit 225fb4e38d6e5ec5eff877be2a84c8fd29af0c3e
Author: Otto Fowler <ottobackwa...@gmail.com>
Date: 2017-05-12T18:12:03Z
Merge remote-tracking branch 'apache/master' into METRON-777
commit 97c1be132c114a0f3f729e5f9887adb5aa49f582
Author: Otto Fowler <ottobackwa...@gmail.com>
Date: 2017-05-17T02:22:33Z
Merge remote-tracking branch 'apache/master' into METRON-777
commit b73f35fc25982556ad469de409e15c72b0305a8a
Author: Otto Fowler <ottobackwa...@gmail.com>
Date: 2017-05-19T15:33:30Z
Merge remote-tracking branch 'apache/master' into METRON-777
commit eb2062be4c14d0549a6ce2504dbbe29ea5510141
Author: Otto Fowler <ottobackwa...@gmail.com>
Date: 2017-05-22T11:58:50Z
Merge remote-tracking branch 'apache/master' into METRON-777
commit 382a29e18f9ab161f0a21d92c1ba3ce6f2601eeb
Author: Otto Fowler <ottobackwa...@gmail.com>
Date: 2017-05-22T12:01:38Z
Merge remote-tracking branch 'apache/master' into METRON-777
commit 90a51deb14078b57966f3ad8e2f7e000cbb74ca3
Author: Jon Zeolla <zeo...@gmail.com>
Date: 2017-05-25T21:52:55Z
Merge branch 'METRON-777' of https://github.com/ottobackwards/metron into
METRON-777
commit 2e66ea397ffc82caf6a76d47ba3651754361dd51
Author: Jon Zeolla <zeo...@gmail.com>
Date: 2017-05-25T22:07:23Z
Trivial documentation changes
commit 0dd8cf8c9ab89b43d543de8b5237010a948540a9
Author: Otto Fowler <ottobackwa...@gmail.com>
Date: 2017-05-26T01:58:05Z
Merge remote-tracking branch 'apache/master' into METRON-777
commit a2634f31d2b16f9ece00dbae49a8b0265f091cfd
Author: Otto Fowler <ottobackwa...@gmail.com>
Date: 2017-05-26T02:00:07Z
Merge branch 'METRON-777' of https://github.com/JonZeolla/metron into
jz-metron-777
----
> Metron Management Support for Parser Extensions
> -----------------------------------------------
>
> Key: METRON-947
> URL: https://issues.apache.org/jira/browse/METRON-947
> Project: Metron
> Issue Type: New Feature
> Reporter: Otto Fowler
> Assignee: Otto Fowler
>
> Building off of METRON-777 and METRON-942, metron should have the ability to
> manage parser extensions from the stellar repl
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
