[
https://issues.apache.org/jira/browse/METRON-1466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16383908#comment-16383908
]
Michael Miklavcic commented on METRON-1466:
-------------------------------------------
Nevermind, this isn't even getting to the indexing topology.
> The Elastic Search index are not getting generated in the latest metron
> deployments for all the topologies
> ----------------------------------------------------------------------------------------------------------
>
> Key: METRON-1466
> URL: https://issues.apache.org/jira/browse/METRON-1466
> Project: Metron
> Issue Type: Bug
> Environment: Metron secured cluster
> Reporter: Mohan
> Priority: Major
> Attachments: bro_worker.log, enrichmnts_worker.log,
> indexing_worker.log
>
>
> The Elastic Search index are not getting generated in the latest metron
> deployments for all the topologies ,
> I see that the data is getting published to the respective topics and also
> getting enriched .
> below are the console consumer output for topics "bro" and "enrichments" I
> dont get anything for "indexing" topic .
> {code:java}
> ./kafka-console-consumer.sh --zookeeper
> ctr-e138-1518143905142-61778-01-000002.hwx.site:2181 --security-protocol
> PLAINTEXTSASL --topic bro
> {"http":{"id.orig_p":49210,"status_code":200,"method":"GET","request_body_len":0,"id.resp_p":80,"uri":"\/picture.php?k=11iqmfg&b7f2a994c3eaaf014608b272c46cf764","tags":[],"uid":"CRDObQRKAmoHCQq1a","referrer":"http:\/\/7oqnsnzwwnm6zb7y.gigapaysun.com\/11iQmfg","resp_mime_types":["image\/png"],"trans_depth":3,"host":"7oqnsnzwwnm6zb7y.gigapaysun.com","status_msg":"OK","id.orig_h":"12.172.138.158","response_body_len":1823,"user_agent":"Mozilla\/4.0
> (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident\/4.0; SLCC2; .NET CLR
> 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC
> 6.0)","ts":1.469686182899762E9,"id.resp_h":"13.163.121.204","resp_fuids":["FQbZZfax5TLVX6M42"]}}
>
> {"http":{"id.orig_p":49210,"status_code":200,"method":"GET","request_body_len":0,"id.resp_p":80,"uri":"\/picture.php?k=11iqmfg&b7f2a994c3eaaf014608b272c46cf764","tags":[],"uid":"CRDObQRKAmoHCQq1a","referrer":"http:\/\/7oqnsnzwwnm6zb7y.gigapaysun.com\/11iQmfg","resp_mime_types":["image\/png"],"trans_depth":3,"host":"7oqnsnzwwnm6zb7y.gigapaysun.com","status_msg":"OK","id.orig_h":"12.172.138.158","response_body_len":1823,"user_agent":"Mozilla\/4.0
> (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident\/4.0; SLCC2; .NET CLR
> 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC
> 6.0)","ts":1.469686182899762E9,"id.resp_h":"13.163.121.204","resp_fuids":["FQbZZfax5TLVX6M42"]}}
>
> {"http":{"id.orig_p":49210,"status_code":200,"method":"GET","request_body_len":0,"id.resp_p":80,"uri":"\/picture.php?k=11iqmfg&b7f2a994c3eaaf014608b272c46cf764","tags":[],"uid":"CRDObQRKAmoHCQq1a","referrer":"http:\/\/7oqnsnzwwnm6zb7y.gigapaysun.com\/11iQmfg","resp_mime_types":["image\/png"],"trans_depth":3,"host":"7oqnsnzwwnm6zb7y.gigapaysun.com","status_msg":"OK","id.orig_h":"12.172.138.158","response_body_len":1823,"user_agent":"Mozilla\/4.0
> (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident\/4.0; SLCC2; .NET CLR
> 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC
> 6.0)","ts":1.469686182899762E9,"id.resp_h":"13.163.121.204","resp_fuids":["FQbZZfax5TLVX6M42"]}}
>
> {"http":{"id.orig_p":49210,"status_code":200,"method":"GET","request_body_len":0,"id.resp_p":80,"uri":"\/picture.php?k=11iqmfg&b7f2a994c3eaaf014608b272c46cf764","tags":[],"uid":"CRDObQRKAmoHCQq1a","referrer":"http:\/\/7oqnsnzwwnm6zb7y.gigapaysun.com\/11iQmfg","resp_mime_types":["image\/png"],"trans_depth":3,"host":"7oqnsnzwwnm6zb7y.gigapaysun.com","status_msg":"OK","id.orig_h":"12.172.138.158","response_body_len":1823,"user_agent":"Mozilla\/4.0
> (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident\/4.0; SLCC2; .NET CLR
> 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC
> 6.0)","ts":1.469686182899762E9,"id.resp_h":"13.163.121.204","resp_fuids":["FQbZZfax5TLVX6M42"]}}
>
> {"http":{"id.orig_p":49210,"status_code":200,"method":"GET","request_body_len":0,"id.resp_p":80,"uri":"\/picture.php?k=11iqmfg&b7f2a994c3eaaf014608b272c46cf764","tags":[],"uid":"CRDObQRKAmoHCQq1a","referrer":"http:\/\/7oqnsnzwwnm6zb7y.gigapaysun.com\/11iQmfg","resp_mime_types":["image\/png"],"trans_depth":3,"host":"7oqnsnzwwnm6zb7y.gigapaysun.com","status_msg":"OK","id.orig_h":"12.172.138.158","response_body_len":1823,"user_agent":"Mozilla\/4.0
> (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident\/4.0; SLCC2; .NET CLR
> 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC
> 6.0)","ts":1.469686182899762E9,"id.resp_h":"13.163.121.204","resp_fuids":["FQbZZfax5TLVX6M42"]}}
> ./kafka-console-consumer.sh --zookeeper
> ctr-e138-1518143905142-61778-01-000002.hwx.site:2181 --security-protocol
> PLAINTEXTSASL --topic enrichments
> {"bro_timestamp":"1469686182.899762","status_code":200,"method":"GET","ip_dst_port":80,"request_body_len":0,"uri":"\/picture.php?k=11iqmfg&b7f2a994c3eaaf014608b272c46cf764","tags":[],"source.type":"bro","uid":"CRDObQRKAmoHCQq1a","referrer":"http:\/\/7oqnsnzwwnm6zb7y.gigapaysun.com\/11iQmfg","resp_mime_types":["image\/png"],"trans_depth":3,"protocol":"http","original_string":"HTTP
> | id.orig_p:49210 status_code:200 method:GET request_body_len:0 id.resp_p:80
> uri:\/picture.php?k=11iqmfg&b7f2a994c3eaaf014608b272c46cf764 tags:[]
> uid:CRDObQRKAmoHCQq1a
> referrer:http:\/\/7oqnsnzwwnm6zb7y.gigapaysun.com\/11iQmfg
> resp_mime_types:[\"image\\\/png\"] trans_depth:3
> host:7oqnsnzwwnm6zb7y.gigapaysun.com status_msg:OK id.orig_h:12.172.138.158
> response_body_len:1823 user_agent:Mozilla\/4.0 (compatible; MSIE 8.0; Windows
> NT 6.1; WOW64; Trident\/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729;
> .NET CLR 3.0.30729; Media Center PC 6.0) ts:1469686182.899762
> id.resp_h:13.163.121.204
> resp_fuids:[\"FQbZZfax5TLVX6M42\"]","ip_dst_addr":"13.163.121.204","ip_src_port":49210,"host":"7oqnsnzwwnm6zb7y.gigapaysun.com","status_msg":"OK","guid":"897e534c-ed6b-458c-9b2d-8749a68ed447","response_body_len":1823,"ip_src_addr":"12.172.138.158","user_agent":"Mozilla\/4.0
> (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident\/4.0; SLCC2; .NET CLR
> 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC
> 6.0)","resp_fuids":["FQbZZfax5TLVX6M42"],"timestamp":1469686182899}
> {"bro_timestamp":"1469686182.899762","status_code":200,"method":"GET","ip_dst_port":80,"request_body_len":0,"uri":"\/picture.php?k=11iqmfg&b7f2a994c3eaaf014608b272c46cf764","tags":[],"source.type":"bro","uid":"CRDObQRKAmoHCQq1a","referrer":"http:\/\/7oqnsnzwwnm6zb7y.gigapaysun.com\/11iQmfg","resp_mime_types":["image\/png"],"trans_depth":3,"protocol":"http","original_string":"HTTP
> | id.orig_p:49210 status_code:200 method:GET request_body_len:0 id.resp_p:80
> uri:\/picture.php?k=11iqmfg&b7f2a994c3eaaf014608b272c46cf764 tags:[]
> uid:CRDObQRKAmoHCQq1a
> referrer:http:\/\/7oqnsnzwwnm6zb7y.gigapaysun.com\/11iQmfg
> resp_mime_types:[\"image\\\/png\"] trans_depth:3
> host:7oqnsnzwwnm6zb7y.gigapaysun.com status_msg:OK id.orig_h:12.172.138.158
> response_body_len:1823 user_agent:Mozilla\/4.0 (compatible; MSIE 8.0; Windows
> NT 6.1; WOW64; Trident\/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729;
> .NET CLR 3.0.30729; Media Center PC 6.0) ts:1469686182.899762
> id.resp_h:13.163.121.204
> resp_fuids:[\"FQbZZfax5TLVX6M42\"]","ip_dst_addr":"13.163.121.204","ip_src_port":49210,"host":"7oqnsnzwwnm6zb7y.gigapaysun.com","status_msg":"OK","guid":"6caaab54-e78d-4d31-9b2c-5c2bb25d609f","response_body_len":1823,"ip_src_addr":"12.172.138.158","user_agent":"Mozilla\/4.0
> (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident\/4.0; SLCC2; .NET CLR
> 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC
> 6.0)","resp_fuids":["FQbZZfax5TLVX6M42"],"timestamp":1469686182899}
> {"bro_timestamp":"1469686182.899762","status_code":200,"method":"GET","ip_dst_port":80,"request_body_len":0,"uri":"\/picture.php?k=11iqmfg&b7f2a994c3eaaf014608b272c46cf764","tags":[],"source.type":"bro","uid":"CRDObQRKAmoHCQq1a","referrer":"http:\/\/7oqnsnzwwnm6zb7y.gigapaysun.com\/11iQmfg","resp_mime_types":["image\/png"],"trans_depth":3,"protocol":"http","original_string":"HTTP
> | id.orig_p:49210 status_code:200 method:GET request_body_len:0 id.resp_p:80
> uri:\/picture.php?k=11iqmfg&b7f2a994c3eaaf014608b272c46cf764 tags:[]
> uid:CRDObQRKAmoHCQq1a
> referrer:http:\/\/7oqnsnzwwnm6zb7y.gigapaysun.com\/11iQmfg
> resp_mime_types:[\"image\\\/png\"] trans_depth:3
> host:7oqnsnzwwnm6zb7y.gigapaysun.com status_msg:OK id.orig_h:12.172.138.158
> response_body_len:1823 user_agent:Mozilla\/4.0 (compatible; MSIE 8.0; Windows
> NT 6.1; WOW64; Trident\/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729;
> .NET CLR 3.0.30729; Media Center PC 6.0) ts:1469686182.899762
> id.resp_h:13.163.121.204
> resp_fuids:[\"FQbZZfax5TLVX6M42\"]","ip_dst_addr":"13.163.121.204","ip_src_port":49210,"host":"7oqnsnzwwnm6zb7y.gigapaysun.com","status_msg":"OK","guid":"a7475614-789a-4221-88cb-0734ef5fdc0a","response_body_len":1823,"ip_src_addr":"12.172.138.158","user_agent":"Mozilla\/4.0
> (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident\/4.0; SLCC2; .NET CLR
> 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC
> 6.0)","resp_fuids":["FQbZZfax5TLVX6M42"],"timestamp":1469686182899}
> {"bro_timestamp":"1469686182.899762","status_code":200,"method":"GET","ip_dst_port":80,"request_body_len":0,"uri":"\/picture.php?k=11iqmfg&b7f2a994c3eaaf014608b272c46cf764","tags":[],"source.type":"bro","uid":"CRDObQRKAmoHCQq1a","referrer":"http:\/\/7oqnsnzwwnm6zb7y.gigapaysun.com\/11iQmfg","resp_mime_types":["image\/png"],"trans_depth":3,"protocol":"http","original_string":"HTTP
> | id.orig_p:49210 status_code:200 method:GET request_body_len:0 id.resp_p:80
> uri:\/picture.php?k=11iqmfg&b7f2a994c3eaaf014608b272c46cf764 tags:[]
> uid:CRDObQRKAmoHCQq1a
> referrer:http:\/\/7oqnsnzwwnm6zb7y.gigapaysun.com\/11iQmfg
> resp_mime_types:[\"image\\\/png\"] trans_depth:3
> host:7oqnsnzwwnm6zb7y.gigapaysun.com status_msg:OK id.orig_h:12.172.138.158
> response_body_len:1823 user_agent:Mozilla\/4.0 (compatible; MSIE 8.0; Windows
> NT 6.1; WOW64; Trident\/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729;
> .NET CLR 3.0.30729; Media Center PC 6.0) ts:1469686182.899762
> id.resp_h:13.163.121.204
> resp_fuids:[\"FQbZZfax5TLVX6M42\"]","ip_dst_addr":"13.163.121.204","ip_src_port":49210,"host":"7oqnsnzwwnm6zb7y.gigapaysun.com","status_msg":"OK","guid":"f05fa5de-5549-4293-85cf-006c66dba5d4","response_body_len":1823,"ip_src_addr":"12.172.138.158","user_agent":"Mozilla\/4.0
> (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident\/4.0; SLCC2; .NET CLR
> 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC
> 6.0)","resp_fuids":["FQbZZfax5TLVX6M42"],"timestamp":1469686182899}
> {"bro_timestamp":"1469686182.899762","status_code":200,"method":"GET","ip_dst_port":80,"request_body_len":0,"uri":"\/picture.php?k=11iqmfg&b7f2a994c3eaaf014608b272c46cf764","tags":[],"source.type":"bro","uid":"CRDObQRKAmoHCQq1a","referrer":"http:\/\/7oqnsnzwwnm6zb7y.gigapaysun.com\/11iQmfg","resp_mime_types":["image\/png"],"trans_depth":3,"protocol":"http","original_string":"HTTP
> | id.orig_p:49210 status_code:200 method:GET request_body_len:0 id.resp_p:80
> uri:\/picture.php?k=11iqmfg&b7f2a994c3eaaf014608b272c46cf764 tags:[]
> uid:CRDObQRKAmoHCQq1a
> referrer:http:\/\/7oqnsnzwwnm6zb7y.gigapaysun.com\/11iQmfg
> resp_mime_types:[\"image\\\/png\"] trans_depth:3
> host:7oqnsnzwwnm6zb7y.gigapaysun.com status_msg:OK id.orig_h:12.172.138.158
> response_body_len:1823 user_agent:Mozilla\/4.0 (compatible; MSIE 8.0; Windows
> NT 6.1; WOW64; Trident\/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729;
> .NET CLR 3.0.30729; Media Center PC 6.0) ts:1469686182.899762
> id.resp_h:13.163.121.204
> resp_fuids:[\"FQbZZfax5TLVX6M42\"]","ip_dst_addr":"13.163.121.204","ip_src_port":49210,"host":"7oqnsnzwwnm6zb7y.gigapaysun.com","status_msg":"OK","guid":"116cb86a-2f41-48a8-8cc0-ee15d25d8db6","response_body_len":1823,"ip_src_addr":"12.172.138.158","user_agent":"Mozilla\/4.0
> (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident\/4.0; SLCC2; .NET CLR
> 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC
> 6.0)","resp_fuids":["FQbZZfax5TLVX6M42"],"timestamp":1469686182899}
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
