[
https://issues.apache.org/jira/browse/METRON-777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16113630#comment-16113630
]
ASF GitHub Bot commented on METRON-777:
---------------------------------------
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/530
```bash
┌─[ottofowler@Winterfell] - [~/tmp] - [Thu Aug 03, 18:36]
└─[$]> mvn archetype:generate -DarchetypeCatalog=local
[INFO] Scanning for projects...
[INFO]
[INFO]
------------------------------------------------------------------------
[INFO] Building Maven Stub Project (No POM) 1
[INFO]
------------------------------------------------------------------------
[INFO]
[INFO] >>> maven-archetype-plugin:3.0.1:generate (default-cli) >
generate-sources @ standalone-pom >>>
[INFO]
[INFO] <<< maven-archetype-plugin:3.0.1:generate (default-cli) <
generate-sources @ standalone-pom <<<
[INFO]
[INFO]
[INFO] --- maven-archetype-plugin:3.0.1:generate (default-cli) @
standalone-pom ---
[INFO] Generating project in Interactive mode
[INFO] No archetype defined. Using maven-archetype-quickstart
(org.apache.maven.archetypes:maven-archetype-quickstart:1.0)
Choose archetype:
1: local -> org.apache.metron:metron-maven-parser-extension-archetype
(Apache Maven Parser Extension Archetype for Metron)
Choose a number or apply filter (format: [groupId:]artifactId, case
sensitive contains): : 1
Define value for property 'groupId': com.michaelmiklavcic
Define value for property 'artifactId': helloparser
Define value for property 'version' 1.0-SNAPSHOT: : 0.4.1
Define value for property 'package' com.michaelmiklavcic: :
Define value for property 'metronVersion': 0.4.1
Define value for property 'parserClassName' (should match expression
'^[A-Z].*$'): Hello
Define value for property 'parserName' (should match expression
'^[a-z]+[A-Z,a-z]+$'): hello
Confirm properties configuration:
groupId: com.michaelmiklavcic
artifactId: helloparser
version: 0.4.1
package: com.michaelmiklavcic
metronVersion: 0.4.1
parserClassName: Hello
parserName: hello
Y: : y
[INFO]
----------------------------------------------------------------------------
[INFO] Using following parameters for creating project from Archetype:
metron-maven-parser-extension-archetype:0.4.1
[INFO]
----------------------------------------------------------------------------
[INFO] Parameter: groupId, Value: com.michaelmiklavcic
[INFO] Parameter: artifactId, Value: helloparser
[INFO] Parameter: version, Value: 0.4.1
[INFO] Parameter: package, Value: com.michaelmiklavcic
[INFO] Parameter: packageInPathFormat, Value: com/michaelmiklavcic
[INFO] Parameter: parserName, Value: hello
[INFO] Parameter: package, Value: com.michaelmiklavcic
[INFO] Parameter: version, Value: 0.4.1
[INFO] Parameter: metronVersion, Value: 0.4.1
[INFO] Parameter: groupId, Value: com.michaelmiklavcic
[INFO] Parameter: parserClassName, Value: Hello
[INFO] Parameter: artifactId, Value: helloparser
[INFO] Parent element not overwritten in
/Users/ottofowler/tmp/helloparser/metron-parser-hello/pom.xml
[INFO] Parent element not overwritten in
/Users/ottofowler/tmp/helloparser/metron-parser-hello-bundle/pom.xml
[INFO] Parent element not overwritten in
/Users/ottofowler/tmp/helloparser/metron-parser-hello-assembly/pom.xml
[INFO] Project created from Archetype in dir:
/Users/ottofowler/tmp/helloparser
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 48.709 s
[INFO] Finished at: 2017-08-03T18:37:13-04:00
[INFO] Final Memory: 15M/245M
[INFO]
------------------------------------------------------------------------
┌─[ottofowler@Winterfell] - [~/tmp] - [Thu Aug 03, 18:37]
└─[$]> tree helloparser
helloparser
├── metron-parser-hello
│ ├── README.md
│ ├── pom.xml
│ └── src
│ ├── main
│ │ ├── config
│ │ │ ├── elasticsearch
│ │ │ │ └── hello_index.template
│ │ │ └── zookeeper
│ │ │ ├── enrichments
│ │ │ │ └── hello.json
│ │ │ ├── indexing
│ │ │ │ └── hello.json
│ │ │ └── parsers
│ │ │ └── hello.json
│ │ ├── java
│ │ │ └── com
│ │ │ └── michaelmiklavcic
│ │ │ └── hello
│ │ │ └── HelloParser.java
│ │ └── resources
│ │ ├── META-INF
│ │ │ ├── LICENSE
│ │ │ └── NOTICE
│ │ └── patterns
│ │ └── common
│ └── test
│ ├── java
│ │ └── com
│ │ └── michaelmiklavcic
│ │ ├── hello
│ │ │ └── HelloParserTest.java
│ │ └── integration
│ │ └── HelloIntegrationTest.java
│ └── resources
│ ├── config
│ │ └── zookeeper
│ │ ├── bundle.properties
│ │ └── global.json
│ ├── data
│ │ ├── parsed
│ │ │ └── test.parsed
│ │ └── raw
│ │ └── test.raw
│ └── log4j.properties
├── metron-parser-hello-assembly
│ ├── pom.xml
│ └── src
│ └── main
│ └── assembly
│ └── assembly.xml
├── metron-parser-hello-bundle
│ └── pom.xml
└── pom.xml
33 directories, 21 files
┌─[ottofowler@Winterfell] - [~/tmp] - [Thu Aug 03, 18:37]
└─[$]> cd helloparser
┌─[ottofowler@Winterfell] - [~/tmp/helloparser] - [Thu Aug 03, 18:37]
└─[$]> mvn package -DskipTests
[INFO] Scanning for projects...
[INFO]
------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] helloparser
[INFO] metron-parser-hello
[INFO] metron-parser-hello-bundle
[INFO] metron-parser-hello-assembly
[INFO]
[INFO]
------------------------------------------------------------------------
[INFO] Building helloparser 0.4.1
[INFO]
------------------------------------------------------------------------
[INFO]
[INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce-versions) @
helloparser ---
[INFO]
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default) @ helloparser
---
[INFO] argLine set to
-javaagent:/Users/ottofowler/.m2/repository/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=/Users/ottofowler/tmp/helloparser/target/jacoco.exec
[INFO]
[INFO] --- jacoco-maven-plugin:0.7.9:report (report) @ helloparser ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO]
[INFO]
------------------------------------------------------------------------
[INFO] Building metron-parser-hello 0.4.1
[INFO]
------------------------------------------------------------------------
[INFO]
[INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce-versions) @
metron-parser-hello ---
[INFO]
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default) @
metron-parser-hello ---
[INFO] argLine set to
-javaagent:/Users/ottofowler/.m2/repository/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=/Users/ottofowler/tmp/helloparser/metron-parser-hello/target/jacoco.exec
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
metron-parser-hello ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] skip non existing resourceDirectory
/Users/ottofowler/tmp/helloparser/metron-parser-hello/src/main/patterns
[INFO] Copying 5 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.5.1:compile (default-compile) @
metron-parser-hello ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to
/Users/ottofowler/tmp/helloparser/metron-parser-hello/target/classes
/Users/ottofowler/tmp/helloparser/metron-parser-hello/src/main/java/com/michaelmiklavcic/hello/HelloParser.java:50:
warning: [DefaultCharset] Implicit use of the platform default charset, which
can result in e.g. non-ASCII characters being silently replaced with '?' in
many environments
String originalString = new String(msg);
^
(see http://errorprone.info/bugpattern/DefaultCharset)
Did you mean 'String originalString = new String(msg, UTF_8);' or 'String
originalString = new String(msg, Charset.defaultCharset());'?
1 warning
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources)
@ metron-parser-hello ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 5 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.5.1:testCompile (default-testCompile) @
metron-parser-hello ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 2 source files to
/Users/ottofowler/tmp/helloparser/metron-parser-hello/target/test-classes
warning: No SupportedSourceVersion annotation found on
org.adrianwalker.multilinestring.MultilineProcessor, returning RELEASE_6.
warning: Supported source version 'RELEASE_6' from annotation processor
'org.adrianwalker.multilinestring.MultilineProcessor' less than -source '1.8'
2 warnings
[INFO]
[INFO] --- maven-surefire-plugin:2.18:test (default-test) @
metron-parser-hello ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-surefire-plugin:2.18:test (unit-tests) @
metron-parser-hello ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- jacoco-maven-plugin:0.7.9:report (report) @ metron-parser-hello
---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO]
[INFO] --- maven-jar-plugin:3.0.2:jar (default-jar) @ metron-parser-hello
---
[INFO] Building jar:
/Users/ottofowler/tmp/helloparser/metron-parser-hello/target/metron-parser-hello-0.4.1.jar
[INFO]
[INFO] --- maven-jar-plugin:3.0.2:test-jar (default) @ metron-parser-hello
---
[INFO] Building jar:
/Users/ottofowler/tmp/helloparser/metron-parser-hello/target/metron-parser-hello-0.4.1-tests.jar
[INFO]
[INFO]
------------------------------------------------------------------------
[INFO] Building metron-parser-hello-bundle 0.4.1
[INFO]
------------------------------------------------------------------------
[INFO]
[INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce-versions) @
metron-parser-hello-bundle ---
[INFO]
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default) @
metron-parser-hello-bundle ---
[INFO] argLine set to
-javaagent:/Users/ottofowler/.m2/repository/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=/Users/ottofowler/tmp/helloparser/metron-parser-hello-bundle/target/jacoco.exec
[INFO]
[INFO] --- maven-resources-plugin:3.0.2:resources (default-resources) @
metron-parser-hello-bundle ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/Users/ottofowler/tmp/helloparser/metron-parser-hello-bundle/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.5.1:compile (default-compile) @
metron-parser-hello-bundle ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-resources-plugin:3.0.2:testResources
(default-testResources) @ metron-parser-hello-bundle ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/Users/ottofowler/tmp/helloparser/metron-parser-hello-bundle/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.5.1:testCompile (default-testCompile) @
metron-parser-hello-bundle ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.18:test (default-test) @
metron-parser-hello-bundle ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-surefire-plugin:2.18:test (unit-tests) @
metron-parser-hello-bundle ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- jacoco-maven-plugin:0.7.9:report (report) @
metron-parser-hello-bundle ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO]
[INFO] --- bundles-maven-plugin:0.4.1:bundle (default-bundle) @
metron-parser-hello-bundle ---
[INFO] Copying metron-parser-hello-0.4.1.jar to
/Users/ottofowler/tmp/helloparser/metron-parser-hello-bundle/target/classes/META-INF/bundled-dependencies/metron-parser-hello-0.4.1.jar
[INFO] Building jar:
/Users/ottofowler/tmp/helloparser/metron-parser-hello-bundle/target/metron-parser-hello-bundle-0.4.1.bundle
[INFO]
[INFO]
------------------------------------------------------------------------
[INFO] Building metron-parser-hello-assembly 0.4.1
[INFO]
------------------------------------------------------------------------
[INFO]
[INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce-versions) @
metron-parser-hello-assembly ---
[INFO]
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default) @
metron-parser-hello-assembly ---
[INFO] argLine set to
-javaagent:/Users/ottofowler/.m2/repository/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=/Users/ottofowler/tmp/helloparser/metron-parser-hello-assembly/target/jacoco.exec
[INFO]
[INFO] --- jacoco-maven-plugin:0.7.9:report (report) @
metron-parser-hello-assembly ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO]
[INFO] --- maven-assembly-plugin:2.6:single (make-assembly) @
metron-parser-hello-assembly ---
[WARNING] The assembly descriptor contains a filesystem-root relative
reference, which is not cross platform compatible /config
[WARNING] The assembly descriptor contains a filesystem-root relative
reference, which is not cross platform compatible /patterns
[WARNING] The assembly descriptor contains a filesystem-root relative
reference, which is not cross platform compatible /lib
[INFO] Building tar:
/Users/ottofowler/tmp/helloparser/metron-parser-hello-assembly/target/metron-parser-hello-assembly-0.4.1-archive.tar.gz
[INFO]
------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] helloparser ........................................ SUCCESS [
0.446 s]
[INFO] metron-parser-hello ................................ SUCCESS [
2.776 s]
[INFO] metron-parser-hello-bundle ......................... SUCCESS [
0.122 s]
[INFO] metron-parser-hello-assembly ....................... SUCCESS [
0.251 s]
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 4.360 s
[INFO] Finished at: 2017-08-03T18:37:46-04:00
[INFO] Final Memory: 80M/667M
[INFO]
------------------------------------------------------------------------
┌─[ottofowler@Winterfell] - [~/tmp/helloparser] - [Thu Aug 03, 18:38]
└─[$]> cd ~/src/apache/forks/metron
┌─[ottofowler@Winterfell] - [~/src/apache/forks/metron] - [Thu Aug 03,
18:39]
└─[$]> metron-deployment/scripts/platform-info.sh
Metron 0.4.1
--
* METRON-777
--
--
--
ansible 2.0.0.2
config file =
configured module search path = Default w/o overrides
--
Vagrant 1.8.1
--
Python 2.7.10
--
Apache Maven 3.5.0 (ff8f5e7444045639af65f6095c62210b5713f426;
2017-04-03T15:39:06-04:00)
Maven home: /usr/local/Cellar/maven/3.5.0/libexec
Java version: 1.8.0_121, vendor: Oracle Corporation
Java home:
/Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.12.6", arch: "x86_64", family: "mac"
--
Docker version 17.06.0-ce, build 02c1d87
--
node
v6.10.2
--
npm
3.10.10
--
Darwin Winterfell 16.7.0 Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27
PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64 x86_64
--
Total System Memory = 16384 MB
Processor Model: Intel(R) Core(TM) i7-4870HQ CPU
Processor Speed: 2.50GHz
Total Physical Processors: 4
Total cores: 4
Disk information:
/dev/disk1 465Gi 399Gi 66Gi 86% 3906152 4291061127 0% /
This CPU appears to support virtualization
┌─[ottofowler@Winterfell] - [~/src/apache/forks/metron] - [Thu Aug 03,
18:39]
└─[$]>
```
> Create a plugin system for Metron based on 'NAR'
> ------------------------------------------------
>
> Key: METRON-777
> URL: https://issues.apache.org/jira/browse/METRON-777
> Project: Metron
> Issue Type: New Feature
> Reporter: Otto Fowler
> Assignee: Otto Fowler
>
> The success of the Metron project will be greatly dependent on community
> participation, and with that the ability to adapt and extend Metron without
> having to maintain a fork of the project.
> As organizations and individuals look to extend the Metron system with custom
> parsers, enrichments, and stellar functions that may be proprietary in
> nature, the ability to develop and deploy these extensions outside the Metron
> code base is critically important.
> To that end, and after community discussion and proposal we create or
> formalize the 'plugin' development story in Metron.
> The proposal is to adapt the Apache Nifi NAR system for use in Metron. This
> will provide the system with:
> * archetype(s) for developer projects and independent development
> * defined packaging and metadata for 'plugin' products
> * loading and instantiation with classloader isolation capabilities
> * removing the necessity for shading plugin jars
> These capabilities will also enable other features, such as plugin lifecycle,
> plugin configuration+redeployment, and other things.
> The plugin archetypes and their installation will be a followon
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
