Re: How to customise the streams visible in metron ui

er, or search on > specific source types. > > Simon > > On Sun, 17 May 2020 at 12:13, Hema malini wrote: > >> Hi all, >> >> In the Metron UI, is it possible to show only particular streams or >> devices for users. >> >> Thanks and Regards, >> Hema >> > -- > -- > simon elliston ball > @sireb >

How to customise the streams visible in metron ui

Hi all, In the Metron UI, is it possible to show only particular streams or devices for users. Thanks and Regards, Hema

Re: Using Metron profiler

; MaaS is just one way to host machine learning models on YARN. Metron gives > you the tools to call out to models hosted on MaaS or wherever they may be > hosted through a simple HTTP request. > > > Hope this helps some. > > > On Wed, Apr 1, 2020 at 1:34 AM Hema malin

Using Metron profiler

Hi, I was exploring Metron profiler and Metron MaaS. Is it possible to use different models based on different profiler? For instance , user entity behaviour analytics . How can I apply different end point for each user? I am confused on why we are using MaaS in the parsing . Analytics should be

Re: Vertical scroll bar is missing in metron alerts ui

Hi All, Is it configurable based on data source? Thanks and Regards, Hema On Fri, 13 Mar, 2020, 5:43 PM Hema malini, wrote: > Hi, > > When I added a customized data source , vertical scroll bar is missing in > metron alerts ui. For bro data source , vertical scroll bar is there

Vertical scroll bar is missing in metron alerts ui

Hi, When I added a customized data source , vertical scroll bar is missing in metron alerts ui. For bro data source , vertical scroll bar is there but when I change the data source , scroll bar is missing. Thanks, Hema

Re: Using Metron 0.7.2 in production

re was recently some discussion on kicking off another release, but this >> appears to have stalled as of late. Someone should probably resurrect that >> thread and see if we can get the gears turning again. >> >> On Sun, Mar 8, 2020 at 12:11 PM Hema malini >> wrote: >&

Re: Metron MaaS issue

Goncalo Pedras, Thanks a lot.. I gave more memory .Still it failed. Thanks and Regards, Hema On Mon, 9 Mar, 2020, 2:34 PM Gonçalo Pedras, wrote: > Have you tried increasing your service memory? If you have and it doesn’t > work either, there’s a work around until you find a fix for you MaaS >

Re: Metron Alerts & Elastic search Issue

Hi, Also i properly mapped metron_alert in the index template "metron_alert": { "type": "nested" } On Mon, 9 Mar 2020 at 01:24, Hema malini wrote: > Hi, > > Also when i try to update any alerts , i keep

Fwd: Metron Alerts & Elastic search Issue

Hi, Also when i try to update any alerts , i keep getting this error message. Do i have to configure in Elastic search.Please let me know what i am missing org.apache.metron.rest.RestException: Document does not exist, but is required; -- Forwarded message - From: Hema malini

Metron Alerts & Elastic search Issue

Hi , When i try to edit a record in Metron alerts UI , I am getting below error Metron rest API -api/v1/search/findOne fails. Please help Error log: at org.elasticsearch.ElasticsearchException.innerFromXContent(ElasticsearchException.java:490) ~[metron-elasticsearch-storm-0.7.2-uber.jar:?]

Using Metron 0.7.2 in production

Hi all, I could see official release of Metron is 0.7.1 but when I checked in the master branch 0.7.2 has been checked in. 0.7.1 is 110 commits behind the master. Can I use the code from the master branch. I am able to successfully build it . Thanks and Regards, Hema

Re: Elastic search in metron

; > On Mar 5, 2020, at 7:41 AM, Hema malini wrote: > > > >  > > Hi all, > > > > Does Metron is specific to Elastic search version 5.6 . Can I upgrade > the Elastic search. > > > > Thanks and Regards, > > Hema >

Re: Metron MaaS issue

Hi all, Any help would be highly appreciated. On Thu, 27 Feb, 2020, 7:13 PM Hema malini, wrote: > Hi all, > > In metron can I deploy the model live in MaaS. For eg , if I deploy a > model in the pipeline with v1, and redeploy another version by replacing > the existing one, do

Elastic search in metron

Hi all, Does Metron is specific to Elastic search version 5.6 . Can I upgrade the Elastic search. Thanks and Regards, Hema

Metron MaaS issue

Hi all, In metron can I deploy the model live in MaaS. For eg , if I deploy a model in the pipeline with v1, and redeploy another version by replacing the existing one, do I have to kill the existing yarn job. Also , in MaaS, if it has multiple files , it is not able to read from existing

Metron MaaS Issue

Hi, I am not sure whether i am facing an issue or it's a bug . I try to deploy the sample Maas script in metron it works perfectly. In the model collateral , now i add another python class which loads the trained model pkl filed. I gave the absolute path as well as relative path( using dir path).

Re: Metron MaaS deployment issues

on how can i import other python files which are in the same directory. Thanks and Regards Hema On Tue, 10 Dec, 2019, 8:12 PM Hema malini, wrote: > Hi, > > I am facing the same issue even after increasing the memory.Can you please > help because in the log i could session timeout

Re: How can i send batch of data to MaaS

h as > those that would go into an LSTM. You would then use the triage output > method from the profiler to pass a stream of batches to MaaS. > > Simon > > On Tue, 10 Dec 2019 at 16:16, Hema malini wrote: > >> Thanks Otto for the confirmation. >> >> On Tue, 10 Dec, 201

Re: Metron MaaS deployment issues

Hi, I am facing the same issue even after increasing the memory.Can you please help because in the log i could session timeout . Thanks and Regards, Hema On Tue, 10 Dec, 2019, 12:34 AM Hema malini, wrote: > Hi, > > I followed the instructions in the below link -

How can i send batch of data to MaaS

Hi, Is there any way to pass a batch of data to Metron MaaS. We have some models like LSTM, which requires data to be aggregated and passed to model .Can you please suggest whether is it possible. Thanks and Regards, Hema

Metron MaaS deployment issues

Hi, I followed the instructions in the below link - https://community.cloudera.com/t5/Community-Articles/Metron-Model-as-a-Service-Maas-full-dev-platform/ta-p/247394 and also Metron git .I was able to deploy the sample for a datasource. Now i tried to deploy the pretrained model in pytorch and

Re: Metron with Zeek not working.

lease do find the attached Settings I have I tried on 6667 but it does >>> not work >>> >>> On Fri, Dec 6, 2019 at 12:22 PM Hema malini >>> wrote: >>> >>>> Hi, >>>> >>>> Please use the same listener port . I trie

Re: Metron with Zeek not working.

l.com> wrote: > >> Hi, >> >> Please do find the attached Settings I have I tried on 6667 but it does >> not work >> >> On Fri, Dec 6, 2019 at 12:22 PM Hema malini >> wrote: >> >>> Hi, >>> >>> Please use the same listener port .

Re: Metron with Zeek not working.

7 (For Kafka) I tried both 6667 and 9092 but did not receive data in > kafka. > > > > On Fri, Dec 6, 2019 at 9:48 AM Hema malini > wrote: > >> Hi, >> >> This issue is basically due to Kafka configuration. Check your listener >> port in Kafka configur

Re: Metron with Zeek not working.

Hi, This issue is basically due to Kafka configuration. Check your listener port in Kafka configuration .Hope this helps. Thanks and Regards, Hema On Fri, 6 Dec, 2019, 10:17 AM Hema malini, wrote: > Hi, > > Please check the listener property.Use the same port while consuming. &

Re: Metron with Zeek not working.

Hi, Please check the listener property.Use the same port while consuming. Thanks and Regards, Hema On Fri, 6 Dec, 2019, 9:54 AM Farrukh Naveed Anjum, wrote: > Problem is i am not able to Push data into Kafka.. > > I have 1 Name Node, 2 Data Node (All have Kafka Broker + Zookeeper). > > Even

Re: CSV parser

Hi , Thanks for your reply. Issue got resolved after several restarts. I faced issues like when it is a new index it is taking time to index the data. After restarting to nearly five times, suddenly data appeared in the index. Thanks and Regards, Hema On Wed, 13 Nov, 2019, 12:48 PM Hema malini

Re: Enable optional fields in csv parser

r does not support this either. You may have > to do some normalization work on your data if you need to account for this. > > > > > On November 16, 2019 at 08:49:36, Hema malini (nhemamalin...@gmail.com) > wrote: > > Hi all, > > Is there any way to mark some columns as opt

Enable optional fields in csv parser

Hi all, Is there any way to mark some columns as optional in column mapping in CSV parser. Thanks and Regards, Hema

Re: CSV parser

olumns":{ 'A' : 1, 'B' : 2} } } Thanks and Regards, Hema On Wed, 13 Nov, 2019, 3:56 AM Simon Elliston Ball, < si...@simonellistonball.com> wrote: > Perhaps you could post your config? You should have a dictionary in it > called columns which maps column name to index. &

Re: CSV parser

Yes. I uploaded as mentioned in the document. On Tue, 12 Nov, 2019, 9:31 PM Simon Elliston Ball, < si...@simonellistonball.com> wrote: > Did you upload your configs to zookeeper? > > On Tue, 12 Nov 2019 at 16:00, Hema malini wrote: > >> I referred the document - . >

Re: CSV parser

:23 PM Simon Elliston Ball, < si...@simonellistonball.com> wrote: > You modify the column data in the parser config. I suggest checking the > docs for the csv parser. > > On Tue, 12 Nov 2019 at 15:51, Hema malini wrote: > >> Hi, >> >> I enabled CSV parser a

CSV parser

Hi, I enabled CSV parser and created Kafka topic for CSV. Where i need to configure column metadata. Similarly for json parser where i should enable the json key ,value. Do i need to modify the parser class and redeploy the jar again. Thanks and regards, Hema

Re: Metron parser for firewall

ll find details on that in the parser documentation). If > there are messages that are not covered in ten existing map and patterns, > you should consider submitting a PR. > > Simon > > On Fri, 8 Nov 2019 at 07:58, Hema malini wrote: > >> Hi , >> >>

Re: Metron parser for firewall

wrote: > There is a Cisco ASA parser built into metron. I suggest using that. > > Simon > > On Fri, 8 Nov 2019 at 04:50, Hema malini wrote: > >> Hi, >> Any parser available for firewall logs for Metron. I am trying to >> integrate ciscoasa firewall logs with

Metron parser for firewall

Hi, Any parser available for firewall logs for Metron. I am trying to integrate ciscoasa firewall logs with Metron. Thanks and regards, Hema

Re: Push data from elastic search to Metron alerts

more detail - > https://github.com/apache/metron/tree/master/metron-platform/metron-elasticsearch/metron-elasticsearch-common > > On Wed, Nov 6, 2019 at 8:25 AM Hema malini > wrote: > >> Hi all, >> >> I pushed data to elastic search indices from csv . Log data has

Push data from elastic search to Metron alerts

Hi all, I pushed data to elastic search indices from csv . Log data has been parsed to CSV using customized parser and i am trying to view in metron alerts and apply machine learning on top of those log data(Maas). I pushed bro,snort,yaf logs to Metron .When i tried to push syslog in syslog 5424

Re: CSV formatted data to Metron and deploying ML models

mported into HBase. The machine learning > you can perform using Stellar. > > > > Mind you, I haven’t done it myself (yet), so I’m eager to learn if I got > this wrong. > > > >  > > > > Tom. > > > > *From: *Hema malini > *Reply-To: *"

CSV formatted data to Metron and deploying ML models

Hi, We have logs collected from devices defined in csv format and we need to stream the logs to Metron pipeline and apply our ml models. Tried sending bro logs and able to see jivana dashboard. To which Kafka topic i need to send the CSV formatted logs and visualise in Kibana. How can i deploy

Re: Issues in sending syslog

Hi, Finally able to fix the issue. It is due to FQDN. Thanks and regards, Hema On Mon, 9 Sep, 2019, 9:37 PM Hema malini, wrote: > Nothing is written to error indexing topic on es. I could seethe same > timeout error in storm.can you please help which property I need to

Re: [DISCUSS] How are you using in Metron?

Hi , We are using Metron and trying out some of the features. First of all, the way it interconnects with Hadoop ecosystem is too good. We could find out the data flowafter some days working on it. 1. We are using 1,2,3,4,6,7,8,9,10,11,12,14. We are mainly focussing on Maas. 2. Spark integration

Re: Help deploying in AWS

Hi Eric, I also faced errors when setting up Metron in AWS. You can try installing the cluster using Ambari.I faced issues in playbook due to Ansible version issues. Thanks, Hema On Fri, 13 Sep, 2019, 4:30 PM Otto Fowler, wrote: > I believe that the EC2 script has been tested before every

Re: Issues in sending syslog

Miklavcic, wrote: > Also look in the error indexing topic if Solr or ES. And there should also > be data written to HDFS, even in case of error. hdfs dfs -ls > /apps/metron/indexing > > On Wed, Sep 4, 2019 at 10:17 AM Hema malini > wrote: > >> Hi, >> >> I am us

Re: Kafka error in metron

still be > a problem with the config there. > > On Thu, Sep 5, 2019 at 7:24 AM Hema malini > wrote: > >> That issue got resolved after changing the property listeners to >> PLAINTEXT://:9092. >> >> I pushed the logsto bro and syslog.i could see messages only in br

Issues in sending syslog

Hi, I am using Metron 0.7.2. I have setup of three node cluster in centos 7. I am sending syslog and bro logs to corresponding Kafka topic but it is not reaching the bro or syslog storm topology. I could see the messages in the consumer of kafka. But itis neither reaching the enrichment or bro

Re: Kafka error in metron

ault port for Kafka in an HDP install is 6667, not 9092. Also node1 > is the full dev Kafka. You will need to provide a correct bootstrap-server > setting for your brokers. > > Simon > > On Wed, 4 Sep 2019 at 09:12, Hema malini wrote: > >> Hi, >> I installed using Hd

Re: Kafka error in metron

I am able to send messages when I configure listeners properties to single node in Kafka ( for each node,changed the listener property to that host name) and then restarted Kafka from command prompt. How can I manage the same using ambari. Thanks, Hema On Tue, 3 Sep, 2019, 7:04 PM Hema malini

Re: Kafka error in metron

Also, I am able to create topic and see the topics being created. Facing issues while consuming the messages. On Tue, 3 Sep, 2019, 7:00 PM Hema malini, wrote: > Hi, > > I have installed Metron 0.7.2 in three node cluster set up. When running > Kafka consumer in the command prompt g

Kafka error in metron

Hi, I have installed Metron 0.7.2 in three node cluster set up. When running Kafka consumer in the command prompt getting error as "connection to node -1 could not be established.broker may not be available". What should I configure listeners properties in server.properties file.what are all the

Re: Introduction and Slack Invite

Kindly add me as well On Mon, 8 Jul, 2019, 7:57 AM zeo...@gmail.com, wrote: > Welcome to the community. Invites sent for everyone > > On Sun, Jul 7, 2019, 10:00 PM Jesús Daniel Jiménez Paniagua < > jd...@jdani.eu> wrote: > >> If it's possible, same here! >> >> Thanks! >> >> El lun., 8 jul.

Re: Snort logs flow issue

ON/CURRENT/package/scripts/rest_commands.py#L230 > > I believe that table will be empty by default. @Ryan Merriman, do you > happen to know if we currently save user UI data there? > > On Tue, Apr 9, 2019 at 2:14 AM Hema malini > wrote: > >> Hi Michael, >> >>

Re: Snort logs flow issue

Hi Stephanie, Issue got resolved by creating that table in hbase. Thanks and regards, Hema On Tue, Apr 9, 2019, 1:31 PM wrote: > Hello Hema, > > > > Unless I’m wrong, this must be setup in MySQL, the database you use for > Metron REST. > > > > > > *F

Re: Snort logs flow issue

in dev profile. Thanks & Regards Hema On Tue, Apr 9, 2019, 12:44 PM Hema malini wrote: > Hi Michael, > > Thanks for your reply. I couldn't find any errors in metron alerts UI log > . I clicked the search and changed the date range too. Still no records. Do > we have to run

Re: Snort logs flow issue

ard you should be able to see them in the > alerts UI. Any errors in either the alerts UI or REST logs? Also, the new > default behavior is that the UI doesn't initiate a search at login, it's up > to the user to click search. > > On Mon, Apr 8, 2019, 6:38 AM Hema malini wrote: >

Re: Snort logs flow issue

After recreating the index, now we are able to visualize the data in kibana metron dashboard. How we can pass alerts to metron alerts UI. Currently there is no data in alerts UI. How.to configure the logs as alerts On Sat, Apr 6, 2019, 9:21 PM Hema malini wrote: > Sorry for the typo. Can

Re: Snort logs flow issue

Sorry for the typo. Can you please help with the required configuration. On Sat, Apr 6, 2019, 5:39 PM Hema malini wrote: > Are we missing any configuration? Initially elastic search was down. We > figured out the issue and fixed it .Now elastic search is up . We restarted > metron

Re: Snort logs flow issue

into metron alerts dashboard and kibana dashboard..what is the required congratulation On Fri, Apr 5, 2019, 11:52 PM Hema malini wrote: > Sample messages flown in indexing topic > {"msg":"'snort test alert'","parallelenricher.splitter.end.ts":" > 15543

Re: Snort logs flow issue

dapter.end.ts":"1554384503453","ethlen":"0x42","iplen":"53248","adapter. threatinteladapter.begin.ts":"1554384505264","ip_src_port":" 8080","tcpflags":"***A","guid":&q

Re: Snort logs flow issue

-messages 10 > > > On Fri, Apr 5, 2019 at 11:38 AM Hema malini > wrote: > >> We verified it in Storm ui and in Storm topology logs >> >> On Fri, Apr 5, 2019, 10:53 PM Michael Miklavcic < >> michael.miklav...@gmail.com> wrote: >> >>&

Re: Snort logs flow issue

We verified it in Storm ui and in Storm topology logs On Fri, Apr 5, 2019, 10:53 PM Michael Miklavcic wrote: > How did you validate the logs are making it to the indexing topology? > > On Fri, Apr 5, 2019 at 8:12 AM Hema malini > wrote: > >> >> Hi, >> >&g

Snort logs flow issue

Hi, We have installed Metron 0.7.1 in centos 7 using Amabari.Using Nifi we sent the sample snort logs copied from metron git repo to snort kafka topic.We did the same for bro topic.Logs are getting parsed and reached indexing topology . Elastic search indices are not getting created though we

Re: Metron rest db issues

rs > besides 'user' and 'admin' > - don't use the dev profile and manage users yourself > - if you're able to use a more recent version, enable LDAP instead > > Hope that helps. > > On Thu, Mar 14, 2019 at 5:42 AM Hema malini > wrote: > >> I am still facing tge same iss

Re: Metron rest db issues

once upon a command line start of the process. > > Thanks > > Srikanth > > On March 14, 2019 at 2:34 AM Hema malini wrote: > > Hi , > > > > I followed the instructions in > https://docs.hortonworks.com/HDPDocuments/HCP1/HCP-1.3.0/bk_installation/content/installing_

Metron rest db issues

Hi , I followed the instructions in https://docs.hortonworks.com/HDPDocuments/HCP1/HCP-1.3.0/bk_installation/content/installing_rest_app.html to set up the metron rest db . I have used the profile as dev. But whenever I restart the metron rest , I am getting the below error. Caused by:

Metron help on setting up logs

I have installed Metron 0.7 in centos7. After installing the metron i am able to send some messages to the kafka topic for syslog. I have also checked the kafka console and verified that the messages are received. After that I am not able to see those messages in the storm logs or in the console .

Re: Metron rest stops

I am getting 504 gatewaytimed out when I try to login. I have configured MySQL DB with user also added. On Fri, Feb 8, 2019, 1:51 PM Hema malini Hi , > > I installed metron 0.7.1 in centos . I have installed using Ambari. I have > configured metron rest database and configured in

Metron rest stops

Hi , I installed metron 0.7.1 in centos . I have installed using Ambari. I have configured metron rest database and configured in Ambari hi. Metron rest starts successfully for some time and stops after that. How we can debug further Thanks and Regards, Hema

Re: Metron fails to start

; > > > On Thu, Feb 7, 2019 at 11:28 AM Hema malini > wrote: > >> Hi, >> >> I installed Metron 0.7.1 in CentOS 7 with three nodes in EC2 . Metron >> services failing to start. I am getting the below error . As I am setting >> up in cloud public IP keeps

Metron fails to start

Hi, I installed Metron 0.7.1 in CentOS 7 with three nodes in EC2 . Metron services failing to start. I am getting the below error . As I am setting up in cloud public IP keeps changing . Where can i config the public IP for Elastic search server whenever i restart the instances. Is there any

: Metron installation fails with Ambari mpack installation

Hi , I tried to install Metron using mpack in Ambari 2.7.3. I am using Ubuntu 14 ,HDP -2.6.5.0-292 version. I am using only one node and I am getting the below error. Can someone help to fix this error. Traceback (most recent call last): File