Re: [ANNOUNCE] Apache Metron 0.4.0 release

Matt, great job on the release. Thank you much05.07.2017, 13:50, "Justin Leet" :Congrats, everyone!  A lot of people helped out across the board, and I look forward to everyone's contributions moving ahead.On Wed, Jul 5, 2017 at 4:47 PM, Otto Fowler

Re: Netflow Aggregator data into metron pipeline

I agree. We have a parser to consume IPFIX data produced by YAF. If you can get a third-party tool to produce IPFIX data for you we should be able to consume it with our YAF parser.25.06.2017, 06:02, "Nick Allen" :> IMO The best route, would just be the ability to parse netflow

Re: test framework

Hi Moshe, Everything we have is out in the open. Right now we do unit and integration tests and there is a discussion to add an acceptance testing framework, but it has not been designed or implemented yet. If you want to get started then just start with unit and integration tests first. That

Re: Snort

mean that I must start snort from terminal by doing snort -v and then push it to kafka topic? I need to start snort in packet capture mode.On Tue, Oct 10, 2017 at 9:52 PM, James Sirota <jsir...@apache.org> wrote:Yes, you can use Snort. Metron can consume Snort telemetries out of the box. You have

Re: metron dashboard timeout when loads many data

ct 2017 at 21.55 James Sirota <jsir...@apache.org> wrote:You have to restart the ES cluster in a rolling fashion. Meaning restart one data node, then the other, then the other, etc. If you restart them all at once, this will happen11.10.2017, 19:37, "tkg_cangkul" <yuza.ras...@gmail

Re: event correlation on metron

correlation? > > Pls Advice ---  Thank you, James Sirota PMC- Apache Metron jsirota AT apache DOT org

Re: Metron Error in Barematel Installation

i would do mvn clean before re-running as well to make sure your previous artifacts get cleared out16.10.2017, 04:32, "Simon Elliston Ball" :If you can run again with -X and post the debug output somewhere, we should be able to figure out where the dependency that’s

Re: [DISCUSS] Dropping support for elastic 2.x

I am in favor of moving to 5.x and dropping support for 2.x. As Justin mentioned, Elastic have very good docs around cluster migrations and the procedure itself to upgrade from 2.x to 5.x is very simple. https://www.elastic.co/guide/en/elasticsearch/reference/current/restart-upgrade.html I don't

Re: Initial Testing

1 - It us up to you to install and configure snort however you want. Metron simply consumes the Snort telemetry, but is not opinionated about how you setup your sensors. I would recommend starting with the community rule set: https://www.snort.org/faq/what-are-community-rules 2 - Again, this is

Re: who is having problems installing?

possible issues that I will face and how > to solve them > > *Thank you!* > *Caryll* > > On Wed, Oct 4, 2017 at 9:02 AM, Otto Fowler <ottobackwa...@gmail.com> wrote: > >>  Did you mean to send this to users too? >> >>  On October 3, 2017 at 19:12:10, James Sirota

Re: metron dashboard timeout when loads many data

sticsearch 2.3.3 On 10/10/17 23:49, James Sirota wrote: I suspect your Elasticsearch may be in a bad state. If you are using Chrome, can you download the sense plugin and then run the following commands:   GET /_cluster/health?pretty

Re: profiler on metron 0.3.x

There is quite a big feature difference between metron 0.3.x and 0.4.x. I would recommend upgrading even if you have 0.3.x installed.11.11.2017, 02:17, "Youzha" :yes i’ve already configure it. the problem is when i push the profiler.json on metron 0.3.0 and 0.3.1 , there is

Re: ML in Metron

Do you currently have any models we can help you deploy? Thanks,James21.11.2017, 04:44, "Simon Elliston Ball" :Use MaaS: http://metron.apache.org/current-book/metron-analytics/metron-maas-service/index.html On 21 Nov 2017, at 11:43, Syed Hammad Tahir

Re: HDFS SIze

Keep in mind that the VM is not really designed for running real data in production at scale. Why on earth would you need that many disks on a VM? Thanks,James 17.11.2017, 00:02, "Aaron Harris" :Yes I did mean to add in more disks to your VM, but if you already have

Re: Monit and sensor stubs

Hi, any way you can use the full dev platform? https://github.com/apache/metron/tree/master/metron-deployment/vagrant/full-dev-platform16.11.2017, 05:06, "Syed Hammad Tahir" :Hi, I re deployed single node ambari based metron cluster and this time withansibleSkipTags=

Re: Failing to build metron 0.4.1 on CentOS7 (metron-config fails to build)

Really? My build works. My node version is v8.8.0 and npm version 5.4.226.11.2017, 10:27, "Pawel Bialasiewicz" : I am having the same error for 0.4.0 and 0.4.1. I tried different versions of Node and NPM. I also tried building it using the Metron

Re: Unsubscribe

Please follow the following instructions to unsubscribe from the list.http://apache.org/foundation/mailinglists.html 06.12.2017, 03:33, "varsha mordi" :Please remove the username from list.-- Thanks & Regards,Varsha MordiProdevans Technologies LLP.M: +91 9637109734  | 

Re: Install Failed: Metron 0.5.0-rc2 as an MPack - Failed at "package metron-common"

The 'Nothing to do' error typically means yum can't find the package it is trying to install, which could mean your local repo is not configured correctly  can you run yum repolist and make sure you have the metron repo available  20.06.2018, 17:08, "Ahmed Shah" : Hello, I'm trying to

Re: Recommendation around production deployment of Metron on Cisco UCS

What kind of Cisco UCS hardware do you have? The recommendations would be vastly different based on what you have.27.06.2018, 09:46, "deepak kumar" :Hi All,Is there any document around best practices for installing Metron in production environment?What i am looking at is given the number of events

Re: Flush profiler based on condition and not based time

The profile periods are mergeable so an event that occurs over several profile periods can be detected. The profiler flushes according the specified time periods. The idea is to build a historical baseline and compare against it. What you are proposing is a deterministic rule, which we are trying

Re: [DISCUSS] Deprecating metron-api

+1 to deprecate29.06.2018, 14:39, "Ryan Merriman" :Adding user list.  Is anyone out there currently using the metron-api module to query pcap data?On Fri, Jun 29, 2018 at 4:35 PM, Casey Stella wrote:I have no objection and would consider it to be a prerequisite to bringing in

Re: Stellar on another platform?

Out of curiosity, why do you want to build your own SIEM and then bolt Stellar on top? Why not just use Metron, since it has Stellar baked in.17.01.2018, 15:34, "Ian Abreu" : Hey all,   We’ve come across the design decision where we’d like to use Metron tooling as a

Re: Metron User Community Meeting Call

ons >>  > >> - >>  > >> >>  > >> Proposed Feature demonstrations >>  > >> - >>  > >> >>  > >> Community feedback >>  > >> >>  > >> These meetings are *not* for : >>  > >> >>  > >> - >>  > >> >>  > >> Support discussions. Those are best left to the mailing lists. >>  > >> - >>  > >> >>  > >> Development discussions. There is another type of meeting for that. >>  > >> >>  > >> >>  > >> >>  > >> >>  > > >>  > > -- >>  > > >>  > > Jon >>  > > >>  > >> >>  -- >>  Thanks, >>  Andrew >> >>  Subscribe to my book: Streaming Data <http://manning.com/psaltis >> <https://urldefense.proofpoint.com/v2/url?u=http-3A__manning.com_psaltis=DwMGaQ=H50I6Bh8SW87d_bXfZP_8g=yeB_CytRmKpr9adMUN0qfcwJfnmWAQuHY9inQHsSRow=1J5p3hWBZj3Fc4Xy-CytnTi_kafYqRMsY-Ntvr5HlHw=0bpm_zlFmlsG6c8Syr9cEsdZrkKhIuV1mwuJypUBIls=>> >>  <https://www.linkedin.com/pub/andrew-psaltis/1/17b/306 >> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.linkedin.com_pub_andrew-2Dpsaltis_1_17b_306=DwMGaQ=H50I6Bh8SW87d_bXfZP_8g=yeB_CytRmKpr9adMUN0qfcwJfnmWAQuHY9inQHsSRow=1J5p3hWBZj3Fc4Xy-CytnTi_kafYqRMsY-Ntvr5HlHw=pRAxEAoEHPf7qW3ly5Ye1Cbo2nvjGlUlGx1UBbcRPhs=>> >>  twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata >> <https://urldefense.proofpoint.com/v2/url?u=http-3A__twitter.com_intent_user-3Fscreen-5Fname-3Ditmdata=DwMGaQ=H50I6Bh8SW87d_bXfZP_8g=yeB_CytRmKpr9adMUN0qfcwJfnmWAQuHY9inQHsSRow=1J5p3hWBZj3Fc4Xy-CytnTi_kafYqRMsY-Ntvr5HlHw=8ckuW3QNgrz1rYI6eu3yiH09eLd-Msdwyk7CJ13wWMU=>> ---  Thank you, James Sirota PMC- Apache Metron jsirota AT apache DOT org

Re: Metron 0.4.3 (bug)

I have never seen this before. Usually when people experience UI issues of this kind there is an underlying REST service problem. This may be the case here as well, but the UI at least comes up. I've never seen it hang. Does your swagger UI come up ok? Are there any exceptions in the REST logs?

Re: Metron User Community Meeting Call

Hi Guys, Looks like something went wrong with the recording today. It just recorded two short clips:  https://youtu.be/VWMMam4RXFshttps://youtu.be/oxZSVELckzA Daniel, would you mind jumping back on zoom at some point and re-recording that demo? It was awesome and I would like for the community to

Re: Connection Error 111

Gaurav, looks like you have a massively unhealthy Hadoop cluster. I would recommend running our full dev platform as a reference. Please follow the instructions here: https://github.com/apache/metron/tree/master/metron-deployment#how-do-i-deploy-metron-on-a-single-vm29.12.2017, 08:04, "Gaurav

Re: Google Cloud Platform

Not to my knowledge. Are you trying it?24.07.2018, 22:19, "Kevin Waterson" :Has anybody been able to deploy Metron using GCP?ThanksKevin --- Thank you, James SirotaPMC- Apache Metronjsirota AT apache DOT org

Re: Metron Not Reading From Kafka?

Did you mix and match versions of Metron?  Looks like you are having a classpath issue here11.09.2018, 12:01, "David McGinnis" :Sorry for the delay. We are finding a new error in the worker.log.err file for the parser Storm topology. It appears that it is trying to log to Ganglia and failing. We

Re: Recommendation around production deployment of Metron on Cisco UCS

respective of hardware i use ?On Wed, Jun 27, 2018, 22:20 James Sirota <jsir...@apache.org> wrote:What kind of Cisco UCS hardware do you have? The recommendations would be vastly different based on what you have.27.06.2018, 09:46, "deepak kumar" <kdq...@gmail.com>:Hi All,I

Re: Metron Not Reading From Kafka?

Was this flag set by default in the Metron install?  Or is that something that you enabled?14.09.2018, 13:52, "David McGinnis" : All, So I think we've found the root cause of this issue,, so I figured I'd report back. It appears that storm itself was not properly configured, and thus it

Re: use another geoIP db for enrichment

What is the motivation behind switching?  From the open source perspective, I would rather not provide a duplicate of a service we already have.  01.04.2019, 11:54, "Nick Allen" :I agree with Tom's comments.  I don't know of any free services that come close to matching what something like Maxmind

Re: enrichment topology is getting killed

There is nothing in this log that indicates a problem.  Are there any errors or warnings upstream from this?01.04.2019, 06:58, "Meenakshi.S" :Hi , When I tried to send bro/yaf/snort related logs I am seeing that the topology is often getting killed and created .. and my logs are not seen after the

Re: Unable to load Custom Stellar functions from HDFS

Have you tried adding the IP instead of hostname?  Does that work?08.04.2019, 21:35, "Athul Parambath" :Hi Michael,Thanks for your reply.Please find the attached global.json files. global.json -  we have pointed to active namenode. It's working fine,global_with_HA_name - We used namenode HA name

Re: Help to implement MAP kind of use case with metron profiler

Are you trying to set up a profile for a peer group and then compare the user to the peer group?  If yes, then you should just set up a profiler for the peer group instead of aggregating all the users.  12.04.2019, 16:07, "Anil Donthireddy" : I am having profilers that keep on storing the

Re: tuning search query on alert UI

Are you using Solr or ES?  there is a different process based on the indexer used.14.08.2019, 09:18, "Youzha" :Hi, is there any ways to optimize search query on Alert UI ?i try to query all data on my alert UI but the proccess run too slow. especially on my first execute search button. sometimes i

Re: Kafka error in metron

+ 1 to what Mike said.  Also, if you could attach any kafka logs that contain any error messages that would be helpful03.09.2019, 08:42, "Michael Miklavcic" :Hi Hema,A couple Q's for you to help narrow this down:How did you got about installing Kafka and the rest of your Hadoop cluster? Is it an

Re: Metron with multi tenancy support

Metron supports multitenancy through the use of metadata in kafka.  You can also make metron milti-tenant by simply running an instance of Metron per different customer.  What is your use case? Thanks,James09.07.2019, 06:02, "DD Donny Lie" :Hello,In what ways Metron has been supporting multi