Re: Drop events from Metron parser

Nifi’s Syslog 5424 support is based on the same library as Metron uses. On May 5, 2020 at 22:02:11, Dima Kovalyov (dimdr...@gmail.com) wrote: Hello Tom, Exactly, NiFi has range of ingest capable processors including Syslog server. - Dima On Tue, May 5, 2020, 20:00 Yerex, Tom wrote: > Hi Dim

Re: conn.log unable to parse in apeche metron

I’m confused at what you are doing here. What parser are you using? grok or bro? The bro parse works on bro JSON output. Your logs don’t look like they are output as JSON, that is why it is failing I would guess. On March 5, 2020 at 08:30:58, updates on tube (abrahamfik...@gmail.com) wrote: #

Re: linux-syslog(centos 7) parsing in apache metron error

above parser and it works. On February 27, 2020 at 09:19:08, updates on tube (abrahamfik...@gmail.com) wrote: but i can't get the parser? On 2020/02/27 12:13:35, Otto Fowler wrote: br/>> Parsing this messages works with the Syslog31164Parser. Maybe you could > use that. > br/&

Re: linux-syslog(centos 7) parsing in apache metron error

Parsing this messages works with the Syslog3164Parser. Maybe you could use that. On February 27, 2020 at 02:03:50, updates on tube (abrahamfik...@gmail.com) wrote: # I really apriciate your quick responses.. please tell us the valid grok patterns for such kind of log ##

Re: linux-syslog(centos 7) parsing in apache metron error

Can you provide an example of a syslog line that fails? Clean of personal data of course. Also what is your parser configuration? On February 25, 2020 at 01:05:00, updates on tube (abrahamfik...@gmail.com) wrote: On 2020/02/24 19:31:36, Michael Miklavcic wrote: br/>> That's how we route erro

RE: [EXTERNAL] Re: zeek metron-bro-plugin-kafka plugin build errors

rom:* Otto Fowler *Sent:* Tuesday, February 11, 2020 7:49 AM *To:* user@metron.apache.org *Subject:* [EXTERNAL] Re: zeek metron-bro-plugin-kafka plugin build errors What version of bro are you using? On February 10, 2020 at 18:20:11, Beneduce, Kristen (kben...@sandia.gov) wrote: Hello,

Re: zeek metron-bro-plugin-kafka plugin build errors

What version of bro are you using? On February 10, 2020 at 18:20:11, Beneduce, Kristen (kben...@sandia.gov) wrote: Hello, I’m trying to configure Metron bro plugin by following instructions here: https://github.com/apache/metron-bro-plugin-kafka/. I’m unable to build the plugin. I built

Re: Mysterious Metron UI screenshot

I added you to slack, look out for the invite On January 8, 2020 at 16:07:28, Dima Kovalyov (dimdr...@gmail.com) wrote: Hello, Metron community, Here are two screenshots from Slideshare: https://www.slideshare.net/hortonworks/combating-phishing-attacks-how-big-data-helps-detect-impersonators

Re: streaming rsyslog metron using asa parser

ember 25, 2019 at 10:47:54, updates on tube (abrahamfik...@gmail.com) > wrote: > > On 2019/12/23 11:25:45, Otto Fowler wrote: > > That doesn’t look like ASA data. > > > https://github.com/apache/metron/blob/master/metron-platform/metron-integration-test/src/main/sample/da

Re: streaming rsyslog metron using asa parser

/23 11:25:45, Otto Fowler wrote: > That doesn’t look like ASA data. > https://github.com/apache/metron/blob/master/metron-platform/metron-integration-test/src/main/sample/data/asa/raw/asa_raw > > Are you trying to do regular syslog, or ASA. > > > > > On December 23, 2019 a

Re: streaming rsyslog metron using asa parser

That doesn’t look like ASA data. https://github.com/apache/metron/blob/master/metron-platform/metron-integration-test/src/main/sample/data/asa/raw/asa_raw Are you trying to do regular syslog, or ASA. On December 23, 2019 at 01:57:38, updates on tube (abrahamfik...@gmail.com) wrote: i was tryi

Re: Feature request: "outputIndexFunction" for Elasticsearch writer

What might even be more interesting would be to have stellar evaluate conditions and set the index based on the evaluation: pseudo: IF ( parser == BRO ) THEN match(FIELD =x) index = y or something On December 19, 2019 at 05:14:01, Vladimir Mikhailov ( v.mikhai...@content-media.ru) wrote: Hi

Re: How can i send batch of data to MaaS

As Metron is a streaming system, it doesn’t send batches as part of normal in flow operation. MAAS is called through stellar, which operates on a per message basis. The batching we *do* have is at the termination of the stream, at the indexing where we batch writes out of the pipeline. This won’t

Re: Metron with Zeek not working.

I don’t think we support newer versions of bro yet i.e. zeek. On December 5, 2019 at 10:31:12, Farrukh Naveed Anjum ( anjum.farr...@gmail.com) wrote: Hi, I am trying to use upgraded version of Bro that is Zeek. I am unable to receive data into Kafka @load packages/metron-bro-plugin-kafka/Apa

Re: metron-bro-plugin-kafka error

Please start a new thread On December 5, 2019 at 02:07:53, Farrukh Naveed Anjum ( anjum.farr...@gmail.com) wrote: I am not receiving data from Bro to Kafka # @load packages/metron-bro-plugin-kafka/Apache/Kafka redef Kafka::logs_to_send = set(SSH::LOG, RDP::LOG, KRB::LOG, SSL::LOG, DHCP::LOG,

Re: Enable optional fields in csv parser

wrote: > Thanks ..will do preprocessing of data.. > > On Sat, 16 Nov, 2019, 9:25 PM Otto Fowler, > wrote: > >> No, there is no way to do this currently. >> >> The parser parses the line into and array of strings that must match the >> size of the columns. &

Re: Enable optional fields in csv parser

No, there is no way to do this currently. The parser parses the line into and array of strings that must match the size of the columns. The underlying opencsv parser does not support this either. You may have to do some normalization work on your data if you need to account for this. On Nove

RE: Invite for Merton slack channel

n you please add me to the slack channel? Best regards, Sanket ------ *From:* Otto Fowler *Sent:* Wednesday, August 21, 2019 11:16 PM *To:* Wan Nabe ; user@metron.apache.org < user@metron.apache.org> *Subject:* Re: Invite for Merton slack channel Done, join the

Re: [ANNOUNCE] Apache Metron-bro-plugin-kafka release 0.3.0

Just a reminder, if you used my script to verify the RC, please comment : https://github.com/apache/metron-bro-plugin-kafka/pull/38 On October 16, 2019 at 17:19:24, Justin Leet (l...@apache.org) wrote: Hi all, I’m pleased to announce the release of Metron 0.3.0! It's been a little while comin

Re: Help deploying in AWS

. On September 13, 2019 at 06:57:30, Otto Fowler (ottobackwa...@gmail.com) wrote: So you are using https://github.com/apache/metron/tree/master/metron-deployment/amazon-ec2 ? On September 12, 2019 at 16:27:43, Eric Jacksch (e...@jacksch.com) wrote: Greetings, I've been trying to deploy i

Re: Help deploying in AWS

So you are using https://github.com/apache/metron/tree/master/metron-deployment/amazon-ec2 ? On September 12, 2019 at 16:27:43, Eric Jacksch (e...@jacksch.com) wrote: Greetings, I've been trying to deploy in AWS to ec2 instances using the playbook. The VPC is created, instances spun up, etc,

Re: [DISCUSS] HDP 3.1 Upgrade and release strategy

If anyone can think of the things that need to be backed up, please comment the jira. On August 27, 2019 at 17:07:20, Otto Fowler (ottobackwa...@gmail.com) wrote: Good idea METRON–2239 [blocker]. On August 27, 2019 at 16:30:13, Simon Elliston Ball ( si...@simonellistonball.com) wrote: You

Re: [DISCUSS] HDP 3.1 Upgrade and release strategy

Good idea METRON–2239 [blocker]. On August 27, 2019 at 16:30:13, Simon Elliston Ball ( si...@simonellistonball.com) wrote: You could always submit a Jira :) On Tue, 27 Aug 2019 at 21:27, Otto Fowler wrote: > You are right, that is much better than backup_metron_configs.sh. > &g

Re: [DISCUSS] HDP 3.1 Upgrade and release strategy

;>> >>> Something worth noting here is that HDP 2.6.5 is quite old and >>> approaching EoL rapidly, so the issue of upgrade is urgent. I am aware of a >>> large number of users who require this upgrade ASAP, and in fact an aware >>> of zero users who wish

Re: [DISCUSS] HDP 3.1 Upgrade and release strategy

;>> >>> Something worth noting here is that HDP 2.6.5 is quite old and >>> approaching EoL rapidly, so the issue of upgrade is urgent. I am aware of a >>> large number of users who require this upgrade ASAP, and in fact an aware >>> of zero users who wish

Re: [DISCUSS] HDP 3.1 Upgrade and release strategy

an aware >> of zero users who wish to remain on HDP 2. >> >> Perhaps those users who want to stay on the old platform can stick their >> hands up and raise concerns, but this move will likely have to happen very >> soon. >> >> Simon >> >> On T

Re: [DISCUSS] HDP 3.1 Upgrade and release strategy

Although we had the discussion, and some great ideas where passed around, I do not believe we came to some kind of consensus on what 1.0 should look like. So that discussion would have to be picked up again so that we could know where we are at, and make it an actual thing if we were going to make

Re: Invite for Merton slack channel

arma On Tue, Aug 6, 2019 at 8:53 PM Otto Fowler wrote: > sure, give it a sec > > > > On August 6, 2019 at 10:09:36, Thiago Rahal Disposti ( > thiago.ra...@kryptus.com) wrote: > > > Can you please add me ? > > thiago.ra...@kryptus.com > > > Thanks. > Th

Re: Adding to metron slack channel

invited, head over to the metron channel On August 13, 2019 at 23:39:35, Mrinal Pande ( mrinal.pa...@st.niituniversity.in) wrote: Hi, Please add me to the metron slack channel. Regards, Mrinal

Re: Invite for Merton slack channel

invited, head over to the metron channel On August 14, 2019 at 05:55:36, R K Sharma (rksu...@gmail.com) wrote: Hi, Could you please add me to Metron Slack channel ? Regards Rinkesh Sharma On Tue, Aug 6, 2019 at 8:53 PM Otto Fowler wrote: > sure, give it a sec > > > &

Re: Invite for Merton slack channel

sure, give it a sec On August 6, 2019 at 10:09:36, Thiago Rahal Disposti ( thiago.ra...@kryptus.com) wrote: Can you please add me ? thiago.ra...@kryptus.com Thanks. Thiago Rahal On Thu, Jul 18, 2019 at 10:44 PM Otto Fowler wrote: > Both of you are all set, join the metron sl

Re: Invite for Merton slack channel

Both of you are all set, join the metron slack channel On July 18, 2019 at 20:15:33, Aman Diwakar (aman.diwa...@gmail.com) wrote: Me too please On Thu, Jul 18, 2019, 12:32 PM Satish Abburi wrote: > > > Can you please add me also. Thanks. > > > > satish.abb...@sstech.us > > > > *From:* "zeo.

Re: batch indexing in JSON format

We could do something like have some other topology or job that kicks off when an HDFS file is closed. So before we start a new file, we “queue” a log to some conversion topology/job whatever or something like that. On July 15, 2019 at 10:04:08, Michael Miklavcic (michael.miklav...@gmail.com)

Re: Built Failed for 0.7.2

gives this error. On Wed, May 22, 2019 at 4:12 PM Otto Fowler wrote: > Thanks! I’ll create the issue > > > On May 22, 2019 at 01:42:15, Farrukh Naveed Anjum (anjum.farr...@gmail.com) > wrote: > > Requires: /bin/bash > Checking for unpackaged file(s): /usr/lib/rpm/check

Re: Built Failed for 0.7.2

Thanks! I’ll create the issue On May 22, 2019 at 01:42:15, Farrukh Naveed Anjum (anjum.farr...@gmail.com) wrote: Requires: /bin/bash Checking for unpackaged file(s): /usr/lib/rpm/check-files /root/BUILDROOT/metron-0.7.2-root error: Installed (but unpackaged) file(s) found: /usr/metron/0.7.2/

Re: Changing/extending the patternMap in a BasicParser

Any parsers like that are limited, and can be improved on, but if they do that all in code, then they must be compiled to be modified. On top of this, if they are the parsers metron ships, because they all come bundled together, you can’t just rebuild that one parser, unless you create a new parser

Re: Issue when trying to load JSON

‘get at’ the inner json to transform it or something, maybe. I don’t mean to say this is a bug in JSONMap either On Thu, Apr 25, 2019 at 11:31 AM Otto Fowler wrote: > I’m not sure about the name, I’m more thinking about the case. > I’m not sure this is an enveloped issue, or a new featu

Re: Issue when trying to load JSON

The issue I think would be with the transformations not working or being applicable no? On April 25, 2019 at 12:19:29, Nick Allen (n...@nickallen.org) wrote: > Stephane: How can I debug this? We created the PARSER* functions to help debug issues like this. Unfortunately, it does not work in thi

Re: Issue when trying to load JSON

Also, our support for nested, unflattened json isn’t great to begin with. Stephane, can you state your use case? Do you want to get _source only to transform it? or do you want to use source as the message and discard the top level fields? other? On April 25, 2019 at 11:31:36, Otto Fowler

Re: Issue when trying to load JSON

: Seems like this would a good additional strategy, something like ENVELOPE_PARSED? Any thoughts on a good name? On Thu, 25 Apr 2019 at 16:20, Otto Fowler wrote: > So, the enveloped message doesn’t support getting an already parsed json > object from the enveloped json, we would have to d

Re: Issue when trying to load JSON

would be inefficient. Can you open a jira with the information you provided? On April 25, 2019 at 11:12:38, Otto Fowler (ottobackwa...@gmail.com) wrote: Raw message in this case assumes that the raw message is a String embedded in the json field that you supply, not a nested json object, so it

Re: Issue when trying to load JSON

Raw message in this case assumes that the raw message is a String embedded in the json field that you supply, not a nested json object, so it is looking for “_source” : “some other embedded string of some format like syslog in json” There are other message strategies, but I’m not sure they would

Re: Help regarding Parser Configuration

t; "adapter:hostfromjsonlistadapter:end:ts": [ > 1551159049014 > ], > "parallelenricher:splitter:end:ts": [ > 1551159049016 > ], > "adapter:threatinteladapter:begin:ts": [ > 1551159049016 > ], > "adapter:geoadapter:en

Re: Help regarding Parser Configuration

How can I extract fields and apply the Parser Chaining in it ? On Wed, Feb 20, 2019 at 10:08 PM Simon Elliston Ball < si...@simonellistonball.com> wrote: > You might like to look into parser chaining for this: > https://metron.apache.org/current-book/metron-platform/metron-parsers/ParserCha

Re: Help regarding Parser Configuration

Can you print what the fields are after parsing? These are the fields that you will be able to use Stellar on, to possibly extract your info. Are you using the Bro parser? On February 20, 2019 at 02:14:17, Farrukh Naveed Anjum ( anjum.farr...@gmail.com) wrote: Hi, I wanted to know how can I def

Re: Unable to use Syslog Parser

;adapter:hostfromjsonlistadapter:begin:ts": [ 1550209569921 ], "parallelenricher:enrich:end:ts": [ 1550209569923 ], "parallelenricher:splitter:begin:ts": [ 1550209569923 ], "adapter:threatinteladapter:end:ts": [

Re: Unable to use Syslog Parser

n Type: 3 This event is generated when a logon session is destroyed. It may be positively correlated with a logon event using the Logon ID value. Logon IDs are only unique between reboots on the same computer. On Wed, Feb 13, 2019 at 7:01 PM Otto Fowler wrote: > Also include the configurati

Re: Unable to use Syslog Parser

logon session is destroyed. It may be positively correlated with a logon event using the Logon ID value. Logon IDs are only unique between reboots on the same computer. On Wed, Feb 13, 2019 at 7:01 PM Otto Fowler wrote: > Also include the configuration of the parser please. > > > > On

Re: Unable to find the paths of YAF

The patterns, if not in HDFS are loaded from the uber jar itself. Can you create a jira with the error and a sanitized version of the failing line, as well as the sensor configuration you have? On February 11, 2019 at 03:48:36, Farrukh Naveed Anjum ( anjum.farr...@gmail.com) wrote: Could not fin

Re: Unable to use Syslog Parser

Also include the configuration of the parser please. On February 13, 2019 at 09:00:08, Otto Fowler (ottobackwa...@gmail.com) wrote: Farrukh, This error means that the syslog line you are passing in is not proper per the spec. Can you create a jira, with this info, and attach or otherwise

Re: Unable to use Syslog Parser

Farrukh, This error means that the syslog line you are passing in is not proper per the spec. Can you create a jira, with this info, and attach or otherwise include a SANITIZED (change IP, machine names, business stuff etc since this will be on the internet ) version of the failing line? I’ll be a

Re: Zeppelin Stellar

Can you open a jira for this? On February 5, 2019 at 11:42:41, Landricombe, Tobin ( tobin.landrico...@roke.co.uk) wrote: Hi, Using Stellar on Metron 0.7.0.0 on HDP 2.6.5.0 on Ambari 2.6.2.0 on CentOS 7, when running... REST_GET('http://test.localdomain/test.json') ...on the REPL, I get... {b

Re: Centos VM Install Fails, Python Exception Syntax

I don’t think the issue is in the VM though. "File "/usr/local/Cellar/ansible/2.7.6/libexec/lib/python3.7/site-packages/ansible/plugins/action/normal.py", line 46, in run” that is a home-brew path. FWIW, https://github.com/apache/metron/pull/1261 is a PR to build the full dev vm using docker to

Re: Centos VM Install Fails, Python Exception Syntax

I think you should have python 2.7.11 at a minimum on the machine running ansible maybe. On February 1, 2019 at 07:55:59, Ryan Sommers (ry...@rpsommers.com) wrote: When attempting to build the single-vm I am getting an error in what appears to be command-line python. I added 'ansible.verbose = "

RE: How to provide hbase-site.xml to Stellar Processor Java API

information. As per my understanding there seems to be no scope to pass my own build Hbase configuration object to execute Stellar queries in my extended API. I may need to re-write lot of things in my extended API in the way Stellar processor works to override the Hbase configuration. @Otto Fowler

Re: How to provide hbase-site.xml to Stellar Processor Java API

ethod: public HTableInterface getTable(Configuration config, String tableName) throws IOException if you implement your own where you ignore the config argument and resolve the hbase table with your own injected config that will work Thanks Mohan DV On 1/24/19, 8:56 PM, "Otto Fowler" wrote

Re: How to provide hbase-site.xml to Stellar Processor Java API

Hi Anil, Can you create a jira on this with these details and a general overview of your use case? It looks like the HbaseConfiguration we use in the HTableConnector is done using the create() method, which creates from resources. I think we would need to do some work to support the external file.

Re: what version metron on HCP 1.8.0

You should post this to the Hortonworks community forum or contact your Hortonworks representative. >%s/Hortonworks/Cloudera/g On January 22, 2019 at 02:05:06, tkg_cangkul (yuza.ras...@gmail.com) wrote: Hi, I've downloaded hcp 1.8.0 mpack from this link : https://docs.hortonworks.com/HDPDocume

Re: Metron - How to use Java API of Profiler client

Hi Anil, Can you create a jira to capture your use case? On January 2, 2019 at 04:41:18, Anil Donthireddy (anil.donthire...@sstech.us) wrote: Hi, As part of our requirements, it will be good if we have an interface to access Metron profiler statistics from other applications developed in Java

Re: Graphs based on Metron or PCAP data

Pieter, Can you create a jira with your use case? It is important to capture. We have some outstanding jira’s around graph support. On January 2, 2019 at 04:40:23, Stefan Kupstaitis-Dunkler ( stefan@gmail.com) wrote: Hi Pieter, Happy new year! I believe that always depends on a lot o

Re: CEF parser timestamp rt field not present

Pieter, You can always create jira issues for things that you think are wrong or missing in the existing parsers, and maybe that work can get done. There are also things ‘in the pipeline’ that you may want to think about. - There is a new regex parser that just landed. - There is a syslog 3164 pa

Re: Metron Upgrade from 0.4.3 to 0.6.0 issues

( mcginn...@avalonconsult.com) wrote: Otto, Can you please add me as well? I've been working with Doug on this migration recently, so I should be able to help answer any questions y'all have. -- David -- *From:* Otto Fowler *Sent:* Thursday, November 29,

Re: Metron Upgrade from 0.4.3 to 0.6.0 issues

I’m going to add you to slack as well. On November 29, 2018 at 19:28:45, Doug Mann (ma...@avalonconsult.com) wrote: Hi all, I've been running into lots of issues regarding an installation of Metron 0.6.0 (upgrading from 0.4.3) failing silently during the deployment phase in Ambari. I've docume

Re: Running MAAS in batch

That may be the best MAAS explanation I’ve seen Simon. On November 16, 2018 at 10:28:57, Simon Elliston Ball ( si...@simonellistonball.com) wrote: MaaS is designed to wrap model inference (scoring) an event at a time, via a REST api. As such, running it batch doesn't make a lot of sense, since e

Re: Syslog parser design using regx

@gmail.com) wrote: Thanks a lot Otto. That covers everything. On Thu, Nov 1, 2018 at 5:16 PM Otto Fowler wrote: > simple-syslog-5424 uses antlr4 instead of regex because I was unable to > find or develop regex’s to single pass parse structured data. If you look > around you’ll

Re: Syslog parser design using regx

simple-syslog-5424 uses antlr4 instead of regex because I was unable to find or develop regex’s to single pass parse structured data. If you look around you’ll find that most platform’s support for 5424 does not handle structured data, and is implemented as regex. The legacy NiFi syslog support,

Re: Syslog parser issue

in the parser page. As a work around to test the syslog parser with this message could you please add the '-' in place of structured data in the message ? I tried few combination and nothing worked. May be I am adding in wrong place. On Wed, Oct 31, 2018 at 1:40 AM Otto Fowler

Re: Syslog parser issue

example ? Just to under stand the working of syslogparser library in detail to extend in future. Also can I filter fields when using BasicISEParser ? I know we can filter message with stellar but can we filter fields ? Like index only interested fields ? On Tue, Oct 30, 2018 at 11:29 PM Otto Fowler

Re: Syslog parser issue

Per the spec which this is written to, if you don’t have structured data, you need to have a ‘-‘ marker. So this is not valid 5424. That is from a cursory look. Metron has a dedicated ISE parser, have you tried that? If you would like to have the parser have a setting to optionally accept missin

Re: Build Errors

You can look at the metron-builder role in metron-deployment/ansible to see how the referenced vagrant machine is built On October 24, 2018 at 11:34:11, Michael Miklavcic ( michael.miklav...@gmail.com) wrote: Hi David, building the RPMs requires building full Metron first. Switch to the root pro

Re: Hello

Irc is kind of dead. I sent you a slack invite. Welcome! On October 21, 2018 at 10:57:04, Scott Cote (scott.c...@lucidworks.com) wrote: Hello All, I’m somewhat new to the Metron Community though I have been exploring it and the source for some time. Looking forward to being a contributor by

Re: Metron dev environments moving to require Ansible 2.4+

;? It was the only reference I could find on the wiki. All of the READMEs should be updated as a part of the PR, but feel free to provide your input if I missed anything. Jon On Fri, Sep 28, 2018 at 10:15 AM Otto Fowler wrote: > We should make sure the non-source documentation is updated

Re: Metron dev environments moving to require Ansible 2.4+

We should make sure the non-source documentation is updated On September 28, 2018 at 09:32:52, zeo...@gmail.com (zeo...@gmail.com) wrote: Hi All, As it currently sits, once METRON-1758 is merged into the code base, Ansible 2.4 or later will be requir

Re: WELCOME to user@metron.apache.org

Invite sent On September 9, 2018 at 07:27:08, siavosh.zarrasv...@gmail.com ( siavosh.zarrasv...@gmail.com) wrote: Hi all, Also, could while I still would like to be added to the slack channel, I wonder if this thread could be deleted as well? Accidentally, I am sending my phone number as part

Re: Add account to slack

Done On September 4, 2018 at 04:13:45, Lehuede sebastien (lehued...@gmail.com) wrote: Hi All, I take the liberty to use Ivan's email to ask for a Slack account to join the channel too. Regards, Sebastien. Le mar. 4 sept. 2018 à 10:02, Ivan Paterno a écrit : > Hi, can i have an account to joi

Re: Add account to slack

Done On September 4, 2018 at 04:02:06, Ivan Paterno (ivan.pate...@elmec.it) wrote: Hi, can i have an account to join the slack channel? Ivan Paterno Security Specialist ivan.pate...@elmec.it Elmec Informatica SPA HQ - via Pret, 1 21020 Brunello (VA) Tel. +39 0332802627 Fax +39 033287

Re: Issue with Enrichment topology: java.lang.OutOfMemoryError: GC overhead limit exceeded

So, before you where doing GEO you did not have the problem? If you took the GEO out it would stop? On August 21, 2018 at 11:04:56, Anil Donthireddy (anil.donthire...@sstech.us) wrote: Hi, We have been keep on getting the error “java.lang.OutOfMemoryError: GC overhead limit exceeded” at Enri

Re: Google Cloud Platform

I would also recommend creating a jira for the support of metron deployment to GCP, as a peer deployment to the EC2. With some of the requirements for such support On August 9, 2018 at 09:29:48, Justin Leet (justinjl...@gmail.com) wrote: Unfortunately, I have no familiarity with GCP at all, but

Re: CEF Parser not Indexing data via Nifi (SysLogs)

at 10:26 AM Otto Fowler wrote: > Metron does not have a generic Syslog Parser. > > Nifi has Syslog parsing ( either Records or standard Processor ), in two > modes. > > ParseSyslog is the original, where regex’s are used to parse the syslog > RFC3164 and RFC5424, but only

Re: CEF Parser not Indexing data via Nifi (SysLogs)

Metron does not have a generic Syslog Parser. Nifi has Syslog parsing ( either Records or standard Processor ), in two modes. ParseSyslog is the original, where regex’s are used to parse the syslog RFC3164 and RFC5424, but only extracts the common fields ( so the ‘additional info’ like program id

Re: Parser Error while Snort IDS usage

Forgot to put the default format in. It is : private static String defaultDateFormat = "MM/dd/yy-HH:mm:ss.SS"; On June 28, 2018 at 10:06:08, Otto Fowler (ottobackwa...@gmail.com) wrote: The snort parser by default supports dates in the following format: Your dates are missi

Re: Parser Error while Snort IDS usage

The snort parser by default supports dates in the following format: Your dates are missing the ‘yy’. If I add that, your failing message parses: /** 06/28/18-02:06:18.667820 ,1,384,5,"ICMP PING",ICMP,92.222.186.1,,158.69.118.104,,00:FF:FF:FF:FF:FD,0C:C4:7A:79:83:7C,0x3C,,10,8,1,32,32768,8,

Re: How to delete the original message field once the message parsed?

x or cache which does not > have to be quite so complete, where we could definitely optimise down some > of the fields. > > Simon > > On 26 Jun 2018, at 04:57, Otto Fowler wrote: > > Also, theoretically, ‘not throwing anything away’ allows future > processing/reprocessi

Re: How to delete the original message field once the message parsed?

Also, theoretically, ‘not throwing anything away’ allows future processing/reprocessing of data to gain new insights. It is not uncommon from the SEIM’s that I’ve seen to store the raw log information for the reasons Simon states for example. So all these things that Simon and James have mention

java-grok awakening

I have been in contact with the maintainer of java-grok about the status of the project and I am happy to say that there has been activity today, as well as some steps to move it forward and pull some forks back in. https://groups.google.com/forum/#!forum/java-grokhas been created to discuss t

Re: DataWorks Summit San Jose

Sometimes I try a different browser if that happens. Also if you are using ghostery or something that can do it. On February 8, 2018 at 14:15:48, pele_smk (pele...@gmail.com) wrote: Hey Jon, I'm trying to submit my abstract, but it seems the datasummit website submission is broken. It's just sp

Re: CentOS and Ubuntu

The Ubuntu support in Apache Metron is new. Really new. At the moment, developers are not going to be required to test things on Ubuntu when submitting or committing pull requests. Work is also ongoing to get the Ambari install complete. The Ubuntu support should be considered experimental at t

Re: Location of Quickstart "full dev platform"

https://github.com/apache/metron/blob/master/CONTRIBUTING.md On February 6, 2018 at 17:01:09, Jack Hamm (jack.h...@gigamon.com) wrote: Thank you, Ryan! -jack On 2/6/18, 1:56 PM, "Ryan Merriman" wrote: https://github.com/apache/metron/tree/master/metron-deployment/development/centos6

Re: Define a function that can be used in Stellar

I think if we understand the use case, we may be able to think of a more general set of functionality for stellar to meet this and other cases. Will this configuration change? Do you need to track that change without reloading? How *much* is in the configuration? Do we want people putting their

[ANNOUNCE] Metron User Community Meeting

Topic: Community zoom meeting Time: Wednesday, January 31st at 09:30AM PST Join from PC, Mac, Linux, iOS or Android: https://hortonworks.zoom.us/j/658498271 Or join by phone: +1 669 900 6833 (US Toll) or +1 646 558 8656 (US Toll) +1 877

Re: Deployment help needed.

at specified path /Library/Java/JavaVirtualMachines/jdk-9.0.4.jdk/Contents/ >> Home >> > We don’t support Java 9. On January 25, 2018 at 14:16:51, Sujay Jaladi (jsu...@gmail.com) wrote: I deployed a full development environment, started docker and vagrant. It still failed. Attached is the ansibl

Metron User Community Meeting Call

I would like to propose a Metron user community meeting. I propose that we set the meeting next week, and will throw out Wednesday, January 31st at 09:30AM PST, 12:30 on the East Coast and 5:30 in London Towne. This meeting will be held over a web-ex, the details of which will be included in the ac

Re: Deployment help needed.

Can you run metron-deployment/scripts/platform_info.sh and send the output? On January 23, 2018 at 21:43:34, Sujay Jaladi (jsu...@gmail.com) wrote: Hello, Everytime I attempt to deploy apache metron on AWS, I get the following error and all the servers are up and running expect Metron or its co

Re: SysLog using CEF Parser (RSysLogs)

If it reaches the Indexing topology it is not a Parser problem, in almost all cases. On January 22, 2018 at 03:24:35, Farrukh Naveed Anjum ( anjum.farr...@gmail.com) wrote: Yes its Strom Indexing Bolt that is halting it. Any one working on CEF Parser (Can Syslog work with it like RSyslog). We a

Re: Getting Syslogs to Metron

https://metron.apache.org/current-book/metron-platform/metron-indexing/index.html On January 22, 2018 at 02:41:14, Farrukh Naveed Anjum ( anjum.farr...@gmail.com) wrote: Default and (likely) unoptimized writer config used for hdfs writer and sensor profiler

Re: Stellar on another platform?

Fowler (ottobackwa...@gmail.com) wrote: I would also say that you should look at METRON–876 <https://issues.apache.org/jira/browse/METRON-876>. This is the umbrella jira for the effort to separate stellar into a more independent module. On January 18, 2018 at 07:54:38, Otto Fowler (otto

Re: Stellar on another platform?

I would also say that you should look at METRON–876 <https://issues.apache.org/jira/browse/METRON-876>. This is the umbrella jira for the effort to separate stellar into a more independent module. On January 18, 2018 at 07:54:38, Otto Fowler (ottobackwa...@gmail.com) wrote: I have c

Re: Stellar on another platform?

I have created METRON–1409 There are several ways to look at hosting stellar to get examples: - The unit tests - The shell - The storm bolts and transformer classes >From a high level, to host stellar you need to: - Include stellar

Re: Metron Install - Vagrant provision error.

884>* Mobile s...@gandivanetworks.com www.gandivanetworks.com On Jan 17, 2018, at 5:22 PM, Otto Fowler wrote: We do not support Java 9 yet. On January 17, 2018 at 04:25:29, Srikanth Nagarajan (s...@gandivanetworks.com) wrote: InvocationTargetException: java.nio.file.NotDirectoryExc

[ALL] List Replies

The goal of the user list is to foster the Apache Metron community by allowing for common discussion of the uses and application of Apache Metron. The list’s archives also provide a valuable resource for people to look through for ideas and answers to questions. Unless someone specifically reques

  1   2   >