Re: Drop events from Metron parser

Nifi’s Syslog 5424 support is based on the same library as Metron uses. On May 5, 2020 at 22:02:11, Dima Kovalyov (dimdr...@gmail.com) wrote: Hello Tom, Exactly, NiFi has range of ingest capable processors including Syslog server. - Dima On Tue, May 5, 2020, 20:00 Yerex, Tom wrote: > Hi

Re: conn.log unable to parse in apeche metron

I’m confused at what you are doing here. What parser are you using? grok or bro? The bro parse works on bro JSON output. Your logs don’t look like they are output as JSON, that is why it is failing I would guess. On March 5, 2020 at 08:30:58, updates on tube (abrahamfik...@gmail.com) wrote:

Re: linux-syslog(centos 7) parsing in apache metron error

with the above parser and it works. On February 27, 2020 at 09:19:08, updates on tube (abrahamfik...@gmail.com) wrote: but i can't get the parser? On 2020/02/27 12:13:35, Otto Fowler wrote: br/>> Parsing this messages works with the Syslog31164Parser. Maybe you could > use that. > br/>&g

Re: linux-syslog(centos 7) parsing in apache metron error

Parsing this messages works with the Syslog3164Parser. Maybe you could use that. On February 27, 2020 at 02:03:50, updates on tube (abrahamfik...@gmail.com) wrote: # I really apriciate your quick responses.. please tell us the valid grok patterns for such kind of log

Re: linux-syslog(centos 7) parsing in apache metron error

Can you provide an example of a syslog line that fails? Clean of personal data of course. Also what is your parser configuration? On February 25, 2020 at 01:05:00, updates on tube (abrahamfik...@gmail.com) wrote: On 2020/02/24 19:31:36, Michael Miklavcic wrote: br/>> That's how we route

Re: zeek metron-bro-plugin-kafka plugin build errors

What version of bro are you using? On February 10, 2020 at 18:20:11, Beneduce, Kristen (kben...@sandia.gov) wrote: Hello, I’m trying to configure Metron bro plugin by following instructions here: https://github.com/apache/metron-bro-plugin-kafka/. I’m unable to build the plugin. I built

Re: Mysterious Metron UI screenshot

I added you to slack, look out for the invite On January 8, 2020 at 16:07:28, Dima Kovalyov (dimdr...@gmail.com) wrote: Hello, Metron community, Here are two screenshots from Slideshare: https://www.slideshare.net/hortonworks/combating-phishing-attacks-how-big-data-helps-detect-impersonators

Re: streaming rsyslog metron using asa parser

25, 2019 at 10:47:54, updates on tube (abrahamfik...@gmail.com) > wrote: > > On 2019/12/23 11:25:45, Otto Fowler wrote: > > That doesn’t look like ASA data. > > > https://github.com/apache/metron/blob/master/metron-platform/metron-integration-test/src/main/sample/data/as

Re: streaming rsyslog metron using asa parser

/23 11:25:45, Otto Fowler wrote: > That doesn’t look like ASA data. > https://github.com/apache/metron/blob/master/metron-platform/metron-integration-test/src/main/sample/data/asa/raw/asa_raw > > Are you trying to do regular syslog, or ASA. > > > > > On December 23, 2019 a

Re: streaming rsyslog metron using asa parser

That doesn’t look like ASA data. https://github.com/apache/metron/blob/master/metron-platform/metron-integration-test/src/main/sample/data/asa/raw/asa_raw Are you trying to do regular syslog, or ASA. On December 23, 2019 at 01:57:38, updates on tube (abrahamfik...@gmail.com) wrote: i was

Re: Feature request: "outputIndexFunction" for Elasticsearch writer

What might even be more interesting would be to have stellar evaluate conditions and set the index based on the evaluation: pseudo: IF ( parser == BRO ) THEN match(FIELD =x) index = y or something On December 19, 2019 at 05:14:01, Vladimir Mikhailov ( v.mikhai...@content-media.ru) wrote:

Re: Metron with Zeek not working.

I don’t think we support newer versions of bro yet i.e. zeek. On December 5, 2019 at 10:31:12, Farrukh Naveed Anjum ( anjum.farr...@gmail.com) wrote: Hi, I am trying to use upgraded version of Bro that is Zeek. I am unable to receive data into Kafka @load

Re: metron-bro-plugin-kafka error

Please start a new thread On December 5, 2019 at 02:07:53, Farrukh Naveed Anjum ( anjum.farr...@gmail.com) wrote: I am not receiving data from Bro to Kafka # @load packages/metron-bro-plugin-kafka/Apache/Kafka redef Kafka::logs_to_send = set(SSH::LOG, RDP::LOG, KRB::LOG, SSL::LOG, DHCP::LOG,

Re: Enable optional fields in csv parser

wrote: > Thanks ..will do preprocessing of data.. > > On Sat, 16 Nov, 2019, 9:25 PM Otto Fowler, > wrote: > >> No, there is no way to do this currently. >> >> The parser parses the line into and array of strings that must match the >> size of the columns. &

Re: Enable optional fields in csv parser

No, there is no way to do this currently. The parser parses the line into and array of strings that must match the size of the columns. The underlying opencsv parser does not support this either. You may have to do some normalization work on your data if you need to account for this. On

RE: Invite for Merton slack channel

o the slack channel? Best regards, Sanket ------ *From:* Otto Fowler *Sent:* Wednesday, August 21, 2019 11:16 PM *To:* Wan Nabe ; user@metron.apache.org < user@metron.apache.org> *Subject:* Re: Invite for Merton slack channel Done, join the metron channel O

Re: [ANNOUNCE] Apache Metron-bro-plugin-kafka release 0.3.0

Just a reminder, if you used my script to verify the RC, please comment : https://github.com/apache/metron-bro-plugin-kafka/pull/38 On October 16, 2019 at 17:19:24, Justin Leet (l...@apache.org) wrote: Hi all, I’m pleased to announce the release of Metron 0.3.0! It's been a little while

Re: Help deploying in AWS

. On September 13, 2019 at 06:57:30, Otto Fowler (ottobackwa...@gmail.com) wrote: So you are using https://github.com/apache/metron/tree/master/metron-deployment/amazon-ec2 ? On September 12, 2019 at 16:27:43, Eric Jacksch (e...@jacksch.com) wrote: Greetings, I've been trying to deploy in AWS

Re: Help deploying in AWS

So you are using https://github.com/apache/metron/tree/master/metron-deployment/amazon-ec2 ? On September 12, 2019 at 16:27:43, Eric Jacksch (e...@jacksch.com) wrote: Greetings, I've been trying to deploy in AWS to ec2 instances using the playbook. The VPC is created, instances spun up,

Re: [DISCUSS] HDP 3.1 Upgrade and release strategy

If anyone can think of the things that need to be backed up, please comment the jira. On August 27, 2019 at 17:07:20, Otto Fowler (ottobackwa...@gmail.com) wrote: Good idea METRON–2239 [blocker]. On August 27, 2019 at 16:30:13, Simon Elliston Ball ( si...@simonellistonball.com) wrote: You

Re: [DISCUSS] HDP 3.1 Upgrade and release strategy

Good idea METRON–2239 [blocker]. On August 27, 2019 at 16:30:13, Simon Elliston Ball ( si...@simonellistonball.com) wrote: You could always submit a Jira :) On Tue, 27 Aug 2019 at 21:27, Otto Fowler wrote: > You are right, that is much better than backup_metron_configs

Re: [DISCUSS] HDP 3.1 Upgrade and release strategy

gt;> >>> Something worth noting here is that HDP 2.6.5 is quite old and >>> approaching EoL rapidly, so the issue of upgrade is urgent. I am aware of a >>> large number of users who require this upgrade ASAP, and in fact an aware >>> of zero users who wish to r

Re: [DISCUSS] HDP 3.1 Upgrade and release strategy

gt;> >>> Something worth noting here is that HDP 2.6.5 is quite old and >>> approaching EoL rapidly, so the issue of upgrade is urgent. I am aware of a >>> large number of users who require this upgrade ASAP, and in fact an aware >>> of zero users who wish to r

Re: [DISCUSS] HDP 3.1 Upgrade and release strategy

ware >> of zero users who wish to remain on HDP 2. >> >> Perhaps those users who want to stay on the old platform can stick their >> hands up and raise concerns, but this move will likely have to happen very >> soon. >> >> Simon >> >> On Tue, 2

Re: [DISCUSS] HDP 3.1 Upgrade and release strategy

Although we had the discussion, and some great ideas where passed around, I do not believe we came to some kind of consensus on what 1.0 should look like. So that discussion would have to be picked up again so that we could know where we are at, and make it an actual thing if we were going to make

Re: Invite for Merton slack channel

arma On Tue, Aug 6, 2019 at 8:53 PM Otto Fowler wrote: > sure, give it a sec > > > > On August 6, 2019 at 10:09:36, Thiago Rahal Disposti ( > thiago.ra...@kryptus.com) wrote: > > > Can you please add me ? > > thiago.ra...@kryptus.com > > > Thanks. > Th

Re: Adding to metron slack channel

invited, head over to the metron channel On August 13, 2019 at 23:39:35, Mrinal Pande ( mrinal.pa...@st.niituniversity.in) wrote: Hi, Please add me to the metron slack channel. Regards, Mrinal

Re: Invite for Merton slack channel

invited, head over to the metron channel On August 14, 2019 at 05:55:36, R K Sharma (rksu...@gmail.com) wrote: Hi, Could you please add me to Metron Slack channel ? Regards Rinkesh Sharma On Tue, Aug 6, 2019 at 8:53 PM Otto Fowler wrote: > sure, give it a sec > > > &

Re: Invite for Merton slack channel

sure, give it a sec On August 6, 2019 at 10:09:36, Thiago Rahal Disposti ( thiago.ra...@kryptus.com) wrote: Can you please add me ? thiago.ra...@kryptus.com Thanks. Thiago Rahal On Thu, Jul 18, 2019 at 10:44 PM Otto Fowler wrote: > Both of you are all set, join the metron sl

Re: Invite for Merton slack channel

Both of you are all set, join the metron slack channel On July 18, 2019 at 20:15:33, Aman Diwakar (aman.diwa...@gmail.com) wrote: Me too please On Thu, Jul 18, 2019, 12:32 PM Satish Abburi wrote: > > > Can you please add me also. Thanks. > > > > satish.abb...@sstech.us > > > > *From:*

Re: batch indexing in JSON format

We could do something like have some other topology or job that kicks off when an HDFS file is closed. So before we start a new file, we “queue” a log to some conversion topology/job whatever or something like that. On July 15, 2019 at 10:04:08, Michael Miklavcic (michael.miklav...@gmail.com)

Re: Built Failed for 0.7.2

gives this error. On Wed, May 22, 2019 at 4:12 PM Otto Fowler wrote: > Thanks! I’ll create the issue > > > On May 22, 2019 at 01:42:15, Farrukh Naveed Anjum (anjum.farr...@gmail.com) > wrote: > > Requires: /bin/bash > Checking for unpackaged file(s): /usr/lib/rpm/check

Re: Built Failed for 0.7.2

Thanks! I’ll create the issue On May 22, 2019 at 01:42:15, Farrukh Naveed Anjum (anjum.farr...@gmail.com) wrote: Requires: /bin/bash Checking for unpackaged file(s): /usr/lib/rpm/check-files /root/BUILDROOT/metron-0.7.2-root error: Installed (but unpackaged) file(s) found:

Re: Issue when trying to load JSON

ope to ‘get at’ the inner json to transform it or something, maybe. I don’t mean to say this is a bug in JSONMap either On Thu, Apr 25, 2019 at 11:31 AM Otto Fowler wrote: > I’m not sure about the name, I’m more thinking about the case. > I’m not sure this is an enveloped issue, or a new featu

Re: Issue when trying to load JSON

Also, our support for nested, unflattened json isn’t great to begin with. Stephane, can you state your use case? Do you want to get _source only to transform it? or do you want to use source as the message and discard the top level fields? other? On April 25, 2019 at 11:31:36, Otto Fowler

Re: Issue when trying to load JSON

: Seems like this would a good additional strategy, something like ENVELOPE_PARSED? Any thoughts on a good name? On Thu, 25 Apr 2019 at 16:20, Otto Fowler wrote: > So, the enveloped message doesn’t support getting an already parsed json > object from the enveloped json, we would have to d

Re: Issue when trying to load JSON

Raw message in this case assumes that the raw message is a String embedded in the json field that you supply, not a nested json object, so it is looking for “_source” : “some other embedded string of some format like syslog in json” There are other message strategies, but I’m not sure they

Re: Help regarding Parser Configuration

t; "adapter:hostfromjsonlistadapter:end:ts": [ > 1551159049014 > ], > "parallelenricher:splitter:end:ts": [ > 1551159049016 > ], > "adapter:threatinteladapter:begin:ts": [ > 1551159049016 > ], > "adapter:geoadapter:en

Re: Help regarding Parser Configuration

How can I extract fields and apply the Parser Chaining in it ? On Wed, Feb 20, 2019 at 10:08 PM Simon Elliston Ball < si...@simonellistonball.com> wrote: > You might like to look into parser chaining for this: > https://metron.apache.org/current-book/metron-platform/metron-parsers/Pars

Re: Help regarding Parser Configuration

Can you print what the fields are after parsing? These are the fields that you will be able to use Stellar on, to possibly extract your info. Are you using the Bro parser? On February 20, 2019 at 02:14:17, Farrukh Naveed Anjum ( anjum.farr...@gmail.com) wrote: Hi, I wanted to know how can I

Re: Unable to use Syslog Parser

;adapter:hostfromjsonlistadapter:begin:ts": [ 1550209569921 ], "parallelenricher:enrich:end:ts": [ 1550209569923 ], "parallelenricher:splitter:begin:ts": [ 1550209569923 ], "adapter:threatinteladapter:end:ts": [ 155

Re: Unable to use Syslog Parser

n Type: 3 This event is generated when a logon session is destroyed. It may be positively correlated with a logon event using the Logon ID value. Logon IDs are only unique between reboots on the same computer. On Wed, Feb 13, 2019 at 7:01 PM Otto Fowler wrote: > Also include the conf

Re: Unable to find the paths of YAF

The patterns, if not in HDFS are loaded from the uber jar itself. Can you create a jira with the error and a sanitized version of the failing line, as well as the sensor configuration you have? On February 11, 2019 at 03:48:36, Farrukh Naveed Anjum ( anjum.farr...@gmail.com) wrote: Could not

Re: Unable to use Syslog Parser

Also include the configuration of the parser please. On February 13, 2019 at 09:00:08, Otto Fowler (ottobackwa...@gmail.com) wrote: Farrukh, This error means that the syslog line you are passing in is not proper per the spec. Can you create a jira, with this info, and attach or otherwise

Re: Unable to use Syslog Parser

Farrukh, This error means that the syslog line you are passing in is not proper per the spec. Can you create a jira, with this info, and attach or otherwise include a SANITIZED (change IP, machine names, business stuff etc since this will be on the internet ) version of the failing line? I’ll be

Re: Centos VM Install Fails, Python Exception Syntax

I don’t think the issue is in the VM though. "File "/usr/local/Cellar/ansible/2.7.6/libexec/lib/python3.7/site-packages/ansible/plugins/action/normal.py", line 46, in run” that is a home-brew path. FWIW, https://github.com/apache/metron/pull/1261 is a PR to build the full dev vm using docker to

Re: Centos VM Install Fails, Python Exception Syntax

I think you should have python 2.7.11 at a minimum on the machine running ansible maybe. On February 1, 2019 at 07:55:59, Ryan Sommers (ry...@rpsommers.com) wrote: When attempting to build the single-vm I am getting an error in what appears to be command-line python. I added 'ansible.verbose =

RE: How to provide hbase-site.xml to Stellar Processor Java API

information. As per my understanding there seems to be no scope to pass my own build Hbase configuration object to execute Stellar queries in my extended API. I may need to re-write lot of things in my extended API in the way Stellar processor works to override the Hbase configuration. @Otto Fowler

Re: How to provide hbase-site.xml to Stellar Processor Java API

: public HTableInterface getTable(Configuration config, String tableName) throws IOException if you implement your own where you ignore the config argument and resolve the hbase table with your own injected config that will work Thanks Mohan DV On 1/24/19, 8:56 PM, "Otto Fowler" wrote: Hi

Re: How to provide hbase-site.xml to Stellar Processor Java API

Hi Anil, Can you create a jira on this with these details and a general overview of your use case? It looks like the HbaseConfiguration we use in the HTableConnector is done using the create() method, which creates from resources. I think we would need to do some work to support the external

Re: what version metron on HCP 1.8.0

You should post this to the Hortonworks community forum or contact your Hortonworks representative. >%s/Hortonworks/Cloudera/g On January 22, 2019 at 02:05:06, tkg_cangkul (yuza.ras...@gmail.com) wrote: Hi, I've downloaded hcp 1.8.0 mpack from this link :

Re: Metron - How to use Java API of Profiler client

Hi Anil, Can you create a jira to capture your use case? On January 2, 2019 at 04:41:18, Anil Donthireddy (anil.donthire...@sstech.us) wrote: Hi, As part of our requirements, it will be good if we have an interface to access Metron profiler statistics from other applications developed in

Re: Graphs based on Metron or PCAP data

Pieter, Can you create a jira with your use case? It is important to capture. We have some outstanding jira’s around graph support. On January 2, 2019 at 04:40:23, Stefan Kupstaitis-Dunkler ( stefan@gmail.com) wrote: Hi Pieter, Happy new year! I believe that always depends on a lot

Re: CEF parser timestamp rt field not present

Pieter, You can always create jira issues for things that you think are wrong or missing in the existing parsers, and maybe that work can get done. There are also things ‘in the pipeline’ that you may want to think about. - There is a new regex parser that just landed. - There is a syslog 3164

Re: Metron Upgrade from 0.4.3 to 0.6.0 issues

I’m going to add you to slack as well. On November 29, 2018 at 19:28:45, Doug Mann (ma...@avalonconsult.com) wrote: Hi all, I've been running into lots of issues regarding an installation of Metron 0.6.0 (upgrading from 0.4.3) failing silently during the deployment phase in Ambari. I've

Re: Syslog parser design using regx

@gmail.com) wrote: Thanks a lot Otto. That covers everything. On Thu, Nov 1, 2018 at 5:16 PM Otto Fowler wrote: > simple-syslog-5424 uses antlr4 instead of regex because I was unable to > find or develop regex’s to single pass parse structured data. If you look > around yo

Re: Syslog parser design using regx

simple-syslog-5424 uses antlr4 instead of regex because I was unable to find or develop regex’s to single pass parse structured data. If you look around you’ll find that most platform’s support for 5424 does not handle structured data, and is implemented as regex. The legacy NiFi syslog support,

Re: Syslog parser issue

Per the spec which this is written to, if you don’t have structured data, you need to have a ‘-‘ marker. So this is not valid 5424. That is from a cursory look. Metron has a dedicated ISE parser, have you tried that? If you would like to have the parser have a setting to optionally accept

Re: Build Errors

You can look at the metron-builder role in metron-deployment/ansible to see how the referenced vagrant machine is built On October 24, 2018 at 11:34:11, Michael Miklavcic ( michael.miklav...@gmail.com) wrote: Hi David, building the RPMs requires building full Metron first. Switch to the root

Re: Metron dev environments moving to require Ansible 2.4+

ble>? It was the only reference I could find on the wiki. All of the READMEs should be updated as a part of the PR, but feel free to provide your input if I missed anything. Jon On Fri, Sep 28, 2018 at 10:15 AM Otto Fowler wrote: > We should make sure the non-source documentation i

Re: WELCOME to user@metron.apache.org

Invite sent On September 9, 2018 at 07:27:08, siavosh.zarrasv...@gmail.com ( siavosh.zarrasv...@gmail.com) wrote: Hi all, Also, could while I still would like to be added to the slack channel, I wonder if this thread could be deleted as well? Accidentally, I am sending my phone number as part

Re: Add account to slack

Done On September 4, 2018 at 04:13:45, Lehuede sebastien (lehued...@gmail.com) wrote: Hi All, I take the liberty to use Ivan's email to ask for a Slack account to join the channel too. Regards, Sebastien. Le mar. 4 sept. 2018 à 10:02, Ivan Paterno a écrit : > Hi, can i have an account to

Re: Add account to slack

Done On September 4, 2018 at 04:02:06, Ivan Paterno (ivan.pate...@elmec.it) wrote: Hi, can i have an account to join the slack channel? Ivan Paterno Security Specialist ivan.pate...@elmec.it Elmec Informatica SPA HQ - via Pret, 1 21020 Brunello (VA) Tel. +39 0332802627 Fax +39

Re: Issue with Enrichment topology: java.lang.OutOfMemoryError: GC overhead limit exceeded

So, before you where doing GEO you did not have the problem? If you took the GEO out it would stop? On August 21, 2018 at 11:04:56, Anil Donthireddy (anil.donthire...@sstech.us) wrote: Hi, We have been keep on getting the error “java.lang.OutOfMemoryError: GC overhead limit exceeded” at

Re: Google Cloud Platform

I would also recommend creating a jira for the support of metron deployment to GCP, as a peer deployment to the EC2. With some of the requirements for such support On August 9, 2018 at 09:29:48, Justin Leet (justinjl...@gmail.com) wrote: Unfortunately, I have no familiarity with GCP at all,

Re: CEF Parser not Indexing data via Nifi (SysLogs)

:26 AM Otto Fowler wrote: > Metron does not have a generic Syslog Parser. > > Nifi has Syslog parsing ( either Records or standard Processor ), in two > modes. > > ParseSyslog is the original, where regex’s are used to parse the syslog > RFC3164 and RFC5424, but only extra

Re: CEF Parser not Indexing data via Nifi (SysLogs)

Metron does not have a generic Syslog Parser. Nifi has Syslog parsing ( either Records or standard Processor ), in two modes. ParseSyslog is the original, where regex’s are used to parse the syslog RFC3164 and RFC5424, but only extracts the common fields ( so the ‘additional info’ like program

Re: How to delete the original message field once the message parsed?

Also, theoretically, ‘not throwing anything away’ allows future processing/reprocessing of data to gain new insights. It is not uncommon from the SEIM’s that I’ve seen to store the raw log information for the reasons Simon states for example. So all these things that Simon and James have

java-grok awakening

I have been in contact with the maintainer of java-grok about the status of the project and I am happy to say that there has been activity today, as well as some steps to move it forward and pull some forks back in. https://groups.google.com/forum/#!forum/java-grokhas been created to discuss

Re: DataWorks Summit San Jose

Sometimes I try a different browser if that happens. Also if you are using ghostery or something that can do it. On February 8, 2018 at 14:15:48, pele_smk (pele...@gmail.com) wrote: Hey Jon, I'm trying to submit my abstract, but it seems the datasummit website submission is broken. It's just

Re: CentOS and Ubuntu

The Ubuntu support in Apache Metron is new. Really new. At the moment, developers are not going to be required to test things on Ubuntu when submitting or committing pull requests. Work is also ongoing to get the Ambari install complete. The Ubuntu support should be considered experimental at

Re: Location of Quickstart "full dev platform"

https://github.com/apache/metron/blob/master/CONTRIBUTING.md On February 6, 2018 at 17:01:09, Jack Hamm (jack.h...@gigamon.com) wrote: Thank you, Ryan! -jack On 2/6/18, 1:56 PM, "Ryan Merriman" wrote:

Re: Define a function that can be used in Stellar

I think if we understand the use case, we may be able to think of a more general set of functionality for stellar to meet this and other cases. Will this configuration change? Do you need to track that change without reloading? How *much* is in the configuration? Do we want people putting

[ANNOUNCE] Metron User Community Meeting

Topic: Community zoom meeting Time: Wednesday, January 31st at 09:30AM PST Join from PC, Mac, Linux, iOS or Android: https://hortonworks.zoom.us/j/658498271 Or join by phone: +1 669 900 6833 (US Toll) or +1 646 558 8656 (US Toll) +1 877

Re: Deployment help needed.

at specified path /Library/Java/JavaVirtualMachines/jdk-9.0.4.jdk/Contents/ >> Home >> > We don’t support Java 9. On January 25, 2018 at 14:16:51, Sujay Jaladi (jsu...@gmail.com) wrote: I deployed a full development environment, started docker and vagrant. It still failed. Attached is the

Metron User Community Meeting Call

I would like to propose a Metron user community meeting. I propose that we set the meeting next week, and will throw out Wednesday, January 31st at 09:30AM PST, 12:30 on the East Coast and 5:30 in London Towne. This meeting will be held over a web-ex, the details of which will be included in the

Re: Deployment help needed.

Can you run metron-deployment/scripts/platform_info.sh and send the output? On January 23, 2018 at 21:43:34, Sujay Jaladi (jsu...@gmail.com) wrote: Hello, Everytime I attempt to deploy apache metron on AWS, I get the following error and all the servers are up and running expect Metron or its

Re: SysLog using CEF Parser (RSysLogs)

If it reaches the Indexing topology it is not a Parser problem, in almost all cases. On January 22, 2018 at 03:24:35, Farrukh Naveed Anjum ( anjum.farr...@gmail.com) wrote: Yes its Strom Indexing Bolt that is halting it. Any one working on CEF Parser (Can Syslog work with it like RSyslog). We

Re: Stellar on another platform?

Fowler (ottobackwa...@gmail.com) wrote: I would also say that you should look at METRON–876 <https://issues.apache.org/jira/browse/METRON-876>. This is the umbrella jira for the effort to separate stellar into a more independent module. On January 18, 2018 at 07:54:38, Otto Fowler (otto

Re: Stellar on another platform?

I would also say that you should look at METRON–876 <https://issues.apache.org/jira/browse/METRON-876>. This is the umbrella jira for the effort to separate stellar into a more independent module. On January 18, 2018 at 07:54:38, Otto Fowler (ottobackwa...@gmail.com) wrote: I have c

Re: Stellar on another platform?

I have created METRON–1409 There are several ways to look at hosting stellar to get examples: - The unit tests - The shell - The storm bolts and transformer classes >From a high level, to host stellar you need to: - Include

Re: Metron Install - Vagrant provision error.

884>* Mobile s...@gandivanetworks.com www.gandivanetworks.com On Jan 17, 2018, at 5:22 PM, Otto Fowler <ottobackwa...@gmail.com> wrote: We do not support Java 9 yet. On January 17, 2018 at 04:25:29, Srikanth Nagarajan (s...@gandivanetworks.com) wrote: InvocationTar

[ALL] List Replies

The goal of the user list is to foster the Apache Metron community by allowing for common discussion of the uses and application of Apache Metron. The list’s archives also provide a valuable resource for people to look through for ideas and answers to questions. Unless someone specifically

Re: Metron Install - Vagrant provision error.

We do not support Java 9 yet. On January 17, 2018 at 04:25:29, Srikanth Nagarajan (s...@gandivanetworks.com) wrote: InvocationTargetException: java.nio.file.NotDirectoryException: /Library/Java/JavaVirtualMachines/jdk-9.0.1.jdk/Contents/Home/lib/modules

Re: Metron Install - Vagrant provision error.

- Is the the complete error? Can you post the ansible.log in that directory? - Do you have docker installed and running? - can you run METRON_SRC_DIR/metron-deployment/scripts/platform_info.sh and put the output in a mail ottO On January 16, 2018 at 02:42:39, Srikanth Nagarajan

Re: Intro & Question

eton University - cugcr.com <https://cugcr.com/tiki/lce/index.php> -- *From:* Otto Fowler <ottobackwa...@gmail.com> *Sent:* January 9, 2018 11:51 AM *To:* Ahmed Shah; user@metron.apache.org *Subject:* Re: Intro & Question Any interest in submitting

Re: ElasticSearch Indexing not working (Strom Error)

Can we get the complete exception? There may be a ‘caused by’ listing that could help. On January 10, 2018 at 08:53:37, Farrukh Naveed Anjum ( anjum.farr...@gmail.com) wrote: Please some one respond On Mon, Jan 8, 2018 at 1:10 PM, Farrukh Naveed Anjum < anjum.farr...@gmail.com> wrote: >

Re: Installing Metron 0.4.1

Laurens got to it actually. On January 9, 2018 at 11:51:58, Ryan Merriman (merrim...@gmail.com) wrote: Thanks Otto you beat me to it. Was this not added to our documentation? On Tue, Jan 9, 2018 at 10:50 AM, Otto Fowler <ottobackwa...@gmail.com> wrote: > As answered on irc, updatin

Re: Intro & Question

Any interest in submitting this? On January 9, 2018 at 10:42:08, Ahmed Shah (ahmeds...@cmail.carleton.ca) wrote: Hello Srikanth, Our team adapted the Metron 0.4.1 Single Node VM install (Original Code Here:

Re: Installing Metron 0.4.1

As answered on irc, updating gcc got Tarik by this. Laurens FTW! On January 9, 2018 at 11:24:47, Tarik Courdy (tarik.cou...@gmail.com) wrote: Here is the output of the platform_info.sh Thank you. -Tarik On Tue, Jan 9, 2018 at 9:12 AM, Tarik Courdy wrote: > Good

Re: Kafka-Kibana Integration

Please see the Subject:Metron Version thread you started for this. On January 8, 2018 at 02:14:20, Gaurav Bapat (gauravb3...@gmail.com) wrote: Hi, I have deployed Metron on single node but I am not able to visualize logs in Kibana, I have my logs going from NiFi to Kafka topic but I cant see

Re: Metron Version

There are multiple topologies at work to get the data into elasticsearch. The flow is basically: Kafka ( sensor name ) -> parser topology ( sensor name ) -> Kafka (enrichment) -> enrichment topology -> Kafka (indexing) -> indexing topology -> ES + HDFS Each of these topologies are listed in the

Full Dev -> Heartbeat issues

I just started up full dev from the 0.4.2 release tag, and ended up with failed heartbeats for all my services in ambari. After investigation, I found the my /etc/hosts ( on node1 ) had multiple entries for node1 : [vagrant@node1 ~]$ cat /etc/hosts 127.0.0.1 node1 node1 127.0.0.1 localhost ##

Re: [ANNOUNCE] Apache Metron release 0.4.2 and Apache Metron bro plugin for Kafka release 0.1

Thank you Matt, and congratulations everyone! On January 4, 2018 at 16:11:50, Matt Foley (ma...@apache.org) wrote: Metron Community: Happy New Year. I’m happy to announce the release of Metron 0.4.2. A great deal of work from across the community went into this, with over 100 enhancements,

RE: Hello and install issue

Can you run docker? On December 29, 2017 at 22:28:46, James Byrne ( james.by...@intrepidtravel.com) wrote: Can’t run vagrant build as ansible won’t run on windows. For anyone else having the issue, you needs to run mvn package DskipTests in the metron root directory before the deployment

Re: metron vs ossec

Is it in jira? On December 21, 2017 at 10:39:46, Ahmed Shah (ahmeds...@cmail.carleton.ca) wrote: Hello tuutdo, We used OSSEC with OSSIM. My experience with OSSIM is you can't save queries and create elaborate dashboards like you can with Metron. Metron also seems to have a better path for

Re: bro kafka plugin build error on --bro-init=$BRO_SRC option doesn't exist

If you don’t send them through the kafka topic, and use nifi to write to hdfs directly, then you will be skipping the enrichment and ES indexing. Is that what you want? On December 21, 2017 at 06:52:37, Gaurav Bapat (gauravb3...@gmail.com) wrote: Can I send syslogs to HDFS using NiFi without

Re: machine learning libraries supported

Simon, What do you think a good example of python, spark and MaaS would look like? On December 7, 2017 at 07:56:00, Simon Elliston Ball ( si...@simonellistonball.com) wrote: I would recommend starting out with something like Spark, but the short answer is that anything that will run inside a

Re: machine learning libraries supported

Right now, you can look at MaaS, for plugging in machine learning services. If you want to use spark, and you have it on your cluster, you could write your own spark drivers and have them pull from the kakfa topics ( indexing for example ) and run your spark stuff there. On December 7, 2017 at

Re: Basic analysis

again I believe, and this would be a great place for more security people to contribute sample run books for example. There are also efforts by commercial support providers I believe to add more samples of both dashboards and use cases. Simon On 6 Dec 2017, at 14:12, Otto Fowler <ottobac

  1   2   >