[
https://issues.apache.org/jira/browse/METRON-872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15978867#comment-15978867
]
Nick Allen commented on METRON-872:
-----------------------------------
You are probably right [~ottobackwards], it probably has to do with the "Quick
Dev" box image. But at the same time, I am confused why the MPack is looking
for 0.3.1 though. As part of the deployment, I thought we deploy the latest
version of the MPack over top of the "Quick Dev" image. If that is working, I
would expect it to just ignore whatever crud happens to be left in
`/usr/metron/0.3.1`
The problem is definitely not coming from the RPMs that were pushed.
{code}
[root@node1 localrepo]# rpm -qlp /localrepo/*.rpm
/usr/metron
/usr/metron/0.4.0
/usr/metron/0.4.0/bin
/usr/metron/0.4.0/bin/stellar
/usr/metron/0.4.0/bin/zk_load_configs.sh
/usr/metron/0.4.0/lib
/usr/metron/0.4.0/lib/metron-common-0.4.0.jar
/usr/metron
/usr/metron/0.4.0
/usr/metron/0.4.0/bin
/usr/metron/0.4.0/bin/start_management_ui.sh
/usr/metron/0.4.0/web
/usr/metron/0.4.0/web/expressjs
/usr/metron/0.4.0/web/expressjs/package.json
/usr/metron/0.4.0/web/expressjs/server.js
/usr/metron/0.4.0/web/management-ui
/usr/metron/0.4.0/web/management-ui/0.9049e5377e87fd047d2a.chunk.js
/usr/metron/0.4.0/web/management-ui/129c5057f4480f9a353e15e1e1e09f9d.ttf
/usr/metron/0.4.0/web/management-ui/1e5737be5c68c15c3a105b2db9a3b67c.ttf
/usr/metron/0.4.0/web/management-ui/256aab654d3c4dd0e12fd0a32c7e8aa6.ttf
/usr/metron/0.4.0/web/management-ui/3f68500b267c20051088bcc0698af773.ttf
/usr/metron/0.4.0/web/management-ui/5b25afa871e1b896011859f8cdf8da74.ttf
/usr/metron/0.4.0/web/management-ui/674f50d287a8c48dc19ba404d20fe713.eot
/usr/metron/0.4.0/web/management-ui/7b5fb88f12bec8143f00e21bc3222124.ttf
/usr/metron/0.4.0/web/management-ui/893fe14628bd7ac498d550e96367e1be.ttf
/usr/metron/0.4.0/web/management-ui/912ec66d7572ff821749319396470bde.svg
/usr/metron/0.4.0/web/management-ui/ac3f799d5bbaf5196fab15ab8de8431c.ttf
/usr/metron/0.4.0/web/management-ui/af7ae505a9eed503f8b8e6982036873e.woff2
/usr/metron/0.4.0/web/management-ui/assets
/usr/metron/0.4.0/web/management-ui/assets/ace
/usr/metron/0.4.0/web/management-ui/assets/ace/LICENSE
/usr/metron/0.4.0/web/management-ui/assets/ace/ext-language_tools.js
/usr/metron/0.4.0/web/management-ui/assets/ace/mode-grok.js
/usr/metron/0.4.0/web/management-ui/assets/ace/mode-json.js
/usr/metron/0.4.0/web/management-ui/assets/ace/snippets
/usr/metron/0.4.0/web/management-ui/assets/ace/snippets/grok.js
/usr/metron/0.4.0/web/management-ui/assets/ace/snippets/json.js
/usr/metron/0.4.0/web/management-ui/assets/ace/snippets/text.js
/usr/metron/0.4.0/web/management-ui/assets/ace/theme-monokai.js
/usr/metron/0.4.0/web/management-ui/assets/ace/worker-json.js
/usr/metron/0.4.0/web/management-ui/assets/fonts
/usr/metron/0.4.0/web/management-ui/assets/fonts/Roboto
/usr/metron/0.4.0/web/management-ui/assets/fonts/Roboto/LICENSE.txt
/usr/metron/0.4.0/web/management-ui/assets/fonts/Roboto/Roboto-Black.ttf
/usr/metron/0.4.0/web/management-ui/assets/fonts/Roboto/Roboto-BlackItalic.ttf
/usr/metron/0.4.0/web/management-ui/assets/fonts/Roboto/Roboto-Bold.ttf
/usr/metron/0.4.0/web/management-ui/assets/fonts/Roboto/Roboto-BoldItalic.ttf
/usr/metron/0.4.0/web/management-ui/assets/fonts/Roboto/Roboto-Italic.ttf
/usr/metron/0.4.0/web/management-ui/assets/fonts/Roboto/Roboto-Light.ttf
/usr/metron/0.4.0/web/management-ui/assets/fonts/Roboto/Roboto-LightItalic.ttf
/usr/metron/0.4.0/web/management-ui/assets/fonts/Roboto/Roboto-Medium.ttf
/usr/metron/0.4.0/web/management-ui/assets/fonts/Roboto/Roboto-MediumItalic.ttf
/usr/metron/0.4.0/web/management-ui/assets/fonts/Roboto/Roboto-Regular.ttf
/usr/metron/0.4.0/web/management-ui/assets/fonts/Roboto/Roboto-Thin.ttf
/usr/metron/0.4.0/web/management-ui/assets/fonts/Roboto/Roboto-ThinItalic.ttf
/usr/metron/0.4.0/web/management-ui/assets/images
/usr/metron/0.4.0/web/management-ui/assets/images/login.jpg
/usr/metron/0.4.0/web/management-ui/assets/images/logo.png
/usr/metron/0.4.0/web/management-ui/b06871f281fee6b241d60582ae9369b9.ttf
/usr/metron/0.4.0/web/management-ui/b37d0bb73a2f688ecaee01647f41e3e5.ttf
/usr/metron/0.4.0/web/management-ui/d329cc8b34667f114a95422aaad1b063.ttf
/usr/metron/0.4.0/web/management-ui/de74c60991cd63c8b922e0e665a39c7a.ttf
/usr/metron/0.4.0/web/management-ui/favicon.ico
/usr/metron/0.4.0/web/management-ui/fe13e4170719c2fc586501e777bde143.ttf
/usr/metron/0.4.0/web/management-ui/fee66e712a8a08eef5805a46892932ad.woff
/usr/metron/0.4.0/web/management-ui/index.html
/usr/metron/0.4.0/web/management-ui/inline.js
/usr/metron/0.4.0/web/management-ui/license
/usr/metron/0.4.0/web/management-ui/license/LICENSE
/usr/metron/0.4.0/web/management-ui/license/NOTICE
/usr/metron/0.4.0/web/management-ui/main.7641c55ce2dd9d6e32ee.bundle.js
/usr/metron/0.4.0/web/management-ui/main.7641c55ce2dd9d6e32ee.bundle.js.gz
/usr/metron/0.4.0/web/management-ui/scripts.e6c91ee3d771df117c43.bundle.js
/usr/metron/0.4.0/web/management-ui/scripts.e6c91ee3d771df117c43.bundle.js.gz
/usr/metron/0.4.0/web/management-ui/styles.05ca0e9dcfbf9f48a1de.bundle.js
/usr/metron/0.4.0/web/management-ui/styles.05ca0e9dcfbf9f48a1de.bundle.js.gz
/usr/metron
/usr/metron/0.4.0
/usr/metron/0.4.0/bin
/usr/metron/0.4.0/bin/Whois_CSV_to_JSON.py
/usr/metron/0.4.0/bin/flatfile_loader.sh
/usr/metron/0.4.0/bin/geo_enrichment_load.sh
/usr/metron/0.4.0/bin/prune_elasticsearch_indices.sh
/usr/metron/0.4.0/bin/prune_hdfs_files.sh
/usr/metron/0.4.0/bin/threatintel_bulk_prune.sh
/usr/metron/0.4.0/bin/threatintel_taxii_load.sh
/usr/metron/0.4.0/lib
/usr/metron/0.4.0/lib/metron-data-management-0.4.0.jar
/usr/metron
/usr/metron/0.4.0
/usr/metron/0.4.0/bin
/usr/metron/0.4.0/bin/start_elasticsearch_topology.sh
/usr/metron/0.4.0/config
/usr/metron/0.4.0/config/elasticsearch.properties
/usr/metron/0.4.0/lib
/usr/metron/0.4.0/lib/metron-elasticsearch-0.4.0-uber.jar
/usr/metron
/usr/metron/0.4.0
/usr/metron/0.4.0/bin
/usr/metron/0.4.0/bin/latency_summarizer.sh
/usr/metron/0.4.0/bin/start_enrichment_topology.sh
/usr/metron/0.4.0/config
/usr/metron/0.4.0/config/enrichment.properties
/usr/metron/0.4.0/config/zookeeper
/usr/metron/0.4.0/config/zookeeper/enrichments
/usr/metron/0.4.0/config/zookeeper/enrichments/asa.json
/usr/metron/0.4.0/config/zookeeper/enrichments/bro.json
/usr/metron/0.4.0/config/zookeeper/enrichments/snort.json
/usr/metron/0.4.0/config/zookeeper/enrichments/websphere.json
/usr/metron/0.4.0/config/zookeeper/enrichments/yaf.json
/usr/metron/0.4.0/flux
/usr/metron/0.4.0/flux/enrichment
/usr/metron/0.4.0/flux/enrichment/remote.yaml
/usr/metron/0.4.0/lib/metron-enrichment-0.4.0-uber.jar
/usr/metron
/usr/metron/0.4.0
/usr/metron/0.4.0/config/zeppelin/metron/metron-yaf-telemetry.json
/usr/metron/0.4.0/config/zookeeper/indexing/asa.json
/usr/metron/0.4.0/config/zookeeper/indexing/bro.json
/usr/metron/0.4.0/config/zookeeper/indexing/error.json
/usr/metron/0.4.0/config/zookeeper/indexing/snort.json
/usr/metron/0.4.0/config/zookeeper/indexing/websphere.json
/usr/metron/0.4.0/config/zookeeper/indexing/yaf.json
/usr/metron/0.4.0/flux
/usr/metron/0.4.0/flux/indexing
/usr/metron/0.4.0/flux/indexing/remote.yaml
/usr/metron
/usr/metron/0.4.0
/usr/metron/0.4.0/bin
/usr/metron/0.4.0/bin/start_parser_topology.sh
/usr/metron/0.4.0/config
/usr/metron/0.4.0/config/zookeeper
/usr/metron/0.4.0/config/zookeeper/parsers
/usr/metron/0.4.0/config/zookeeper/parsers/asa.json
/usr/metron/0.4.0/config/zookeeper/parsers/bro.json
/usr/metron/0.4.0/config/zookeeper/parsers/jsonMap.json
/usr/metron/0.4.0/config/zookeeper/parsers/snort.json
/usr/metron/0.4.0/config/zookeeper/parsers/squid.json
/usr/metron/0.4.0/config/zookeeper/parsers/websphere.json
/usr/metron/0.4.0/config/zookeeper/parsers/yaf.json
/usr/metron/0.4.0/lib
/usr/metron/0.4.0/lib/metron-parsers-0.4.0-uber.jar
/usr/metron/0.4.0/patterns
/usr/metron/0.4.0/patterns/asa
/usr/metron/0.4.0/patterns/common
/usr/metron/0.4.0/patterns/fireeye
/usr/metron/0.4.0/patterns/sourcefire
/usr/metron/0.4.0/patterns/squid
/usr/metron/0.4.0/patterns/websphere
/usr/metron/0.4.0/patterns/yaf
/usr/metron
/usr/metron/0.4.0
/usr/metron/0.4.0/bin
/usr/metron/0.4.0/bin/pcap_inspector.sh
/usr/metron/0.4.0/bin/pcap_query.sh
/usr/metron/0.4.0/bin/start_pcap_topology.sh
/usr/metron/0.4.0/config
/usr/metron/0.4.0/config/pcap.properties
/usr/metron/0.4.0/flux
/usr/metron/0.4.0/flux/pcap
/usr/metron/0.4.0/flux/pcap/remote.yaml
/usr/metron/0.4.0/lib
/usr/metron/0.4.0/lib/metron-pcap-backend-0.4.0.jar
/usr/metron
/usr/metron/0.4.0
/usr/metron/0.4.0/bin
/usr/metron/0.4.0/bin/start_profiler_topology.sh
/usr/metron/0.4.0/config
/usr/metron/0.4.0/config/profiler.properties
/usr/metron/0.4.0/flux
/usr/metron/0.4.0/flux/profiler
/usr/metron/0.4.0/flux/profiler/remote.yaml
/usr/metron/0.4.0/lib
/usr/metron/0.4.0/lib/metron-profiler-0.4.0-uber.jar
/usr/metron
/usr/metron/0.4.0
/usr/metron/0.4.0/bin
/usr/metron/0.4.0/bin/start_solr_topology.sh
/usr/metron/0.4.0/config
/usr/metron/0.4.0/config/solr.properties
/usr/metron/0.4.0/lib
/usr/metron/0.4.0/lib/metron-solr-0.4.0-uber.jar
{code}
> "Metron Enrichment Start" Fails in Quick Dev
> --------------------------------------------
>
> Key: METRON-872
> URL: https://issues.apache.org/jira/browse/METRON-872
> Project: Metron
> Issue Type: Bug
> Reporter: Nick Allen
>
> h3. Problem
> The "Quick Dev" deployment fails to complete deployment successfully for me.
> Ambari fails to launch the Enrichment topology with this error.
> {code}
> resource_management.core.exceptions.ExecutionFailed: Execution of
> '/usr/metron/0.3.1/bin/zk_load_configs.sh --mode PUSH -i
> /usr/metron/0.3.1/config/zookeeper -z node1:2181' returned 127. /bin/bash:
> /usr/metron/0.3.1/bin/zk_load_configs.sh: No such file or directory
> {code}
> This is confusing to me because we should be on 0.4.0, so why is it
> referencing 0.3.1?
> h3. Steps to Reproduce
> First, I had to `install` the Metron libs (which I don't remember being a
> requirement before.) Then I was able to `vagrant up`.
> {code}
> cd incubator-metron
> mvn clean install -DskipTests -T 1C
> cd metron-deployment/vagrant/quick-dev-platform
> vagrant up
> {code}
> h3. Additional Clues
> When I login to the "Quick Dev" VM, I see that there are some 0.4.0 libs and
> some 0.3.1 libs.
> {code}
> [root@node1 metron]# ls -l /usr/metron/0.3.1/lib/
> total 167800
> -rw-r--r--. 1 root root 85332259 Apr 3 21:05 metron-api-0.3.1.jar
> -rw-r--r--. 1 root root 86490682 Mar 30 13:09 metron-pcap-backend-0.3.1.jar
> [root@node1 metron]# ls -l /usr/metron/0.4.0/lib/
> total 463428
> -rw-r--r--. 1 root root 48425098 Apr 21 13:45 metron-common-0.4.0.jar
> -rw-r--r--. 1 root root 124020704 Apr 21 13:50
> metron-data-management-0.4.0.jar
> -rw-r--r--. 1 root root 114754821 Apr 21 13:50
> metron-elasticsearch-0.4.0-uber.jar
> -rw-r--r--. 1 root root 93280994 Apr 21 13:48
> metron-enrichment-0.4.0-uber.jar
> -rw-r--r--. 1 root root 94059479 Apr 21 13:50 metron-parsers-0.4.0-uber.jar
> {code}
> Where did those 0.3.1 libs come from? When I look in my local Maven
> repository on my build machine, I don't see any 0.3.1 libs at all, only 0.4.0
> {code}
> $ find ~/.m2/repository/org/apache/metron/ -name "0.3.1"
> $ find ~/.m2/repository/org/apache/metron/ -name "0.4.0"
> /Users/nallen/.m2/repository/org/apache/metron//elasticsearch-shaded/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//Metron/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-analytics/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-api/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-common/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-config/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-data-management/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-deployment/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-docker/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-elasticsearch/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-enrichment/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-hbase/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-indexing/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-integration-test/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-interface/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-maas-common/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-maas-service/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-management/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-parsers/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-pcap/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-pcap-backend/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-platform/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-profiler/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-profiler-client/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-profiler-common/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-rest/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-rest-client/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-solr/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-statistics/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-storm-kafka/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-test-utilities/0.4.0
> /Users/nallen/.m2/repository/org/apache/metron//metron-writer/0.4.0
> {code}
> I seem to have the latest "Quick Dev" box image.
> {code}
> $ vagrant box update
> Running with ansible-tags: ["quick_dev", "report"]
> ==> node1: Checking for updates to 'metron/quick_dev'
> node1: Latest installed version: 0.2.0
> node1: Version constraints:
> node1: Provider: virtualbox
> ==> node1: Box 'metron/quick_dev' (v0.2.0) is running the latest version.
> {code}
> And for good measure, here is what my environment looks-like.
> {code}
> $ ./metron-deployment/scripts/platform-info.sh
> Metron 0.4.0
> --
> * master
> --
> commit 41b5b1050cf29e5dcb5d0c36b5a9dbd1cafa745e
> Author: justinjleet <[email protected]>
> Date: Mon Apr 17 15:16:49 2017 -0400
> The MPack should function in a kerberized cluster (justinleet) closes
> apache/incubator-metron#518
> --
> --
> ansible 2.0.0.2
> config file =
> configured module search path = Default w/o overrides
> --
> Vagrant 1.8.1
> --
> Python 2.7.13
> --
> Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5;
> 2015-11-10T11:41:47-05:00)
> Maven home: /usr/local/Cellar/maven/3.3.9/libexec
> Java version: 1.8.0_66, vendor: Oracle Corporation
> Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_66.jdk/Contents/Home/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "mac os x", version: "10.12.4", arch: "x86_64", family: "mac"
> --
> Darwin HW12402.local 16.5.0 Darwin Kernel Version 16.5.0: Fri Mar 3 16:52:33
> PST 2017; root:xnu-3789.51.2~3/RELEASE_X86_64 x86_64
> --
> Total System Memory = 16384 MB
> Processor Model: Intel(R) Core(TM) i7-4870HQ CPU
> Processor Speed: 2.50GHz
> Total Physical Processors: 4
> Total cores: 4
> Disk information:
> /dev/disk1 465Gi 319Gi 145Gi 69% 2694524 4292272755 0% /
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
