Re: [DISCUSS] Slack Channel Use

;> compared to last quarter. That's not to say that we couldn't stand more >>> discussion on the lists, but a lot of the dev discussion happens on >> github >>> and JIRA and I'm happy to see an uptick in user traffic. >>> >>> On Wed, Oct 24, 2018 at 10:

Broken build at the moment

We have a stellar test for date format that is broken because of the daylight savings change. Justin and I have been working through it and I’ll have a PR as soon as my travis build completes. https://issues.apache.org/jira/browse/METRON-1864 Just a heads up that any new builds ( at least in

Re: [DISCUSS] Day 1 User Experience - Getting Metron Running

What is the metron on docker part? On October 26, 2018 at 14:37:48, Nick Allen (n...@nickallen.org) wrote: > Yeah I would +1 katakoda. Has anyone used or have a history with KataKoda? I'd hate to invest time in a hosted solution if the provider isn't going to be around. That's a definite 'con'

metron-elasticsearch integration tests failing after merging in master

https://travis-ci.org/ottobackwards/metron/jobs/445723343 Anyone having ES test problems? Anyone shed any light on this.

Re: [DISCUSS] Slack Channel Use

have never used a search engine and > > > uncovered the answer to my problem in a Slack archive. > > > > > > On Mon, Oct 22, 2018 at 5:05 PM Otto Fowler > > > wrote: > > > > > > > According to Greg Stein, an infra admin o

Re: Invite to Slack Channel

t; > > > > > > > > > On Wed, Oct 17, 2018 at 7:33 PM Michael Miklavcic < > > > > michael.miklav...@gmail.com> wrote: > > > > > > > > > Sent > > > > > On Wed, Oct 17, 2018 at 7:23 AM Tibor Meller <

Re: [DISCUSS] Slack Channel Use

According to Greg Stein, an infra admin on the NiFi slack, the ASF slack that metron is in IS the standard plan, not the free one and is searchable past 10,000 messages. On October 22, 2018 at 15:35:51, Michael Miklavcic ( michael.miklav...@gmail.com) wrote: ...From an archival and broader

Re: [DISCUSS] Slack Channel Use

These questions also occurred on the IRC channel. The difference is that there are more than Jon and I answering now. On October 22, 2018 at 12:18:08, Nick Allen (n...@nickallen.org) wrote: It seems that we are seeing a lot of Metron usage and support questions on the Slack Channel. These are

Re: [DISCUSS] Stellar REST client

I believe the issue of introducing and supporting higher latency enrichments is a systemic one, and should be solved as such, with the rest and other higher latency enrichments build on top of that framework. On October 19, 2018 at 12:22:28, Ryan Merriman (merrim...@gmail.com) wrote: Thanks

Re: Bro plugin unit tests failing

It is INFRA, see INFRA-17091 for example. On October 12, 2018 at 20:47:24, zeo...@gmail.com (zeo...@gmail.com) wrote: So it seems that the last commit before the 0.2 release of metron-bro-plugin-kafka broke the one basic unit test that we had. Since metron 0.6.0 pins to 0.1 <

Re: Custom parser using Jackson instead of json-simple

The ParserBolt is written to JSON simple, so although the interface is in practice it is . The answer is no right now. Feel free to open a jira. On October 5, 2018 at 02:52:37, Muhammed Irshad (irshadkt@gmail.com) wrote: Hi All, Is it not possible to use any Json library other than

Re: Invite to Slack Channel

Done On October 4, 2018 at 05:35:06, Tamás Fodor (ftamas.m...@gmail.com) wrote: Hello, Michael, can you add me as well? Thank you in advance! Tamas On Wed, Oct 3, 2018 at 4:27 PM Michael Miklavcic < michael.miklav...@gmail.com> wrote: > Sent > > On Wed, Oct 3, 2018 at 8:17 AM Shane Ardell

Re: [DISCUSS] Feature Branch guidance

This is all well and good for feature branches, but does nothing for Simon and the type of work he attempted. If we agree that features do not have architectural changes, then we also need to codify how we handle that level of change, assuming anyone is optimistic enough to attempt such a thing in

Re: Metron dev environments moving to require Ansible 2.4+

ble>? It was the only reference I could find on the wiki. All of the READMEs should be updated as a part of the PR, but feel free to provide your input if I missed anything. Jon On Fri, Sep 28, 2018 at 10:15 AM Otto Fowler wrote: > We should make sure the non-source documentation i

Re: Metron dev environments moving to require Ansible 2.4+

We should make sure the non-source documentation is updated On September 28, 2018 at 09:32:52, zeo...@gmail.com (zeo...@gmail.com) wrote: Hi All, As it currently sits, once METRON-1758 is merged into the code base, Ansible 2.4 or later will be

Re: [MENTORS][DISCUSS] LICENSE and NOTICE likely outdated

you share your experience there? On Wed, Sep 12, 2018 at 1:36 PM Otto Fowler wrote: > Are you referring to the dependencies check against the csv? > > > On September 12, 2018 at 13:09:48, Michael Miklavcic ( > michael.miklav...@gmail.com) wrote: > > I'm not sure I fully u

Re: [DISCUSS] Feature branches post-merge

I would drop them. I’ve already clean up FB’s around dead things. On September 6, 2018 at 13:42:55, Michael Miklavcic ( michael.miklav...@gmail.com) wrote: What are we doing with feature branches once they're complete and merged into master? Is our expectation that we'll keep feature branches

Re: IRC Channel -> OPS?

Damn, I was hoping not. It will never happen now On August 29, 2018 at 15:49:26, zeo...@gmail.com (zeo...@gmail.com) wrote: Isn't it Casey? Jon On Wed, Aug 29, 2018, 08:41 Otto Fowler wrote: > Who has ops in the irc channel? > Can you pop in and set the topic to somethin

Re: [DISCUSS] Contributing a General Purpose Regex Parser

I would like to see a PR on this. Do you have an example of a second type of log where this would be useful? Besides something syslog-y? There is a PR out for a Syslog RFC 5424 parser that handles that ( including structured data, which I don’t know if you have in your parser ). What may be more

IRC Channel -> OPS?

Who has ops in the irc channel? Can you pop in and set the topic to something like: “There is an ASF slack with an active metron channel, please email dev@metron.apache.org and request an invite”

Re: [DISCUSS] Getting to a 1.0 release

rate from the code base to allow for an > > > > > > organization that is focused on the user rather than the > > > > implementation. > > > > > > This allows the READMEs to focus on the developer and the > > > > implementation, > > &

package.lock changes during build?

I just did a PR, can saw that the package.lock file for alerts-ui was changed, with updated versions. I did *not* change the file, nor anything in metron-interface. That seems to imply that this file is changed or updated by something that happens during building or deploying full dev. Is this

Re: [DISCUSS] Pcap query branch completion

nt for YARN configuration > > You make a good point. A YARN tuning guide for Metron does sound useful. > I will add a follow on Jira. > > On Mon, Aug 13, 2018 at 4:53 PM, Otto Fowler > wrote: > >> >> - Date range limits on queries >> >> I took the po

Re: [DISCUSS] Getting to a 1.0 release

service api to get at it. I’m all for that too, but think it needs more thought than the ticket captures. Simon On 15 Aug 2018, at 20:53, Otto Fowler wrote: https://issues.apache.org/jira/browse/METRON-343 On August 15, 2018 at 15:47:24, Simon Elliston Ball ( si...@simonellistonball.com) wrote

Re: [DISCUSS] Getting to a 1.0 release

https://issues.apache.org/jira/browse/METRON-106 At least making sure it is met and closing it On August 15, 2018 at 15:53:02, Otto Fowler (ottobackwa...@gmail.com) wrote: https://issues.apache.org/jira/browse/METRON-343 On August 15, 2018 at 15:47:24, Simon Elliston Ball ( si

Re: [DISCUSS] Getting to a 1.0 release

of doing to follow up on the Knox Feature was to add Ranger integration for securing and auditing configs, and potentially extending to the index destinations. Do you think that would cover the secure storage concept? Simon > On 15 Aug 2018, at 20:39, Otto Fowler wrote: > > Secure storage of

Re: [DISCUSS] Getting to a 1.0 release

Secure storage off the top of my head On August 15, 2018 at 14:49:26, zeo...@gmail.com (zeo...@gmail.com) wrote: So, as has been discussed in a few < https://lists.apache.org/thread.html/0445cd8f94dfb844cd5a23ac3eeca04c9f44c9d8f269c6ef12cb3598@%3Cdev.metron.apache.org%3E> other <

Re: [ANNOUNCE] - Apache Metron Slack channel

Done On August 15, 2018 at 14:22:45, Vets, Laurens (laur...@daemon.be) wrote: Could I be invited? On 15-Aug-18 09:48, Michael Miklavcic wrote: > + Metron user list > > On Wed, Aug 15, 2018 at 10:38 AM Michael Miklavcic < > michael.miklav...@gmail.com> wrote: > >> Turns out we are able to

Re: [DISCUSS] Pcap query branch completion

- Date range limits on queries I took the point the wrong way apparently, sorry, I withdraw. I thought you meant allow specifying a limit on the query, not the system imposing a limit. This should be documented with a warning or something - UI should manage a queue/history of jobs I was

Re: [DISCUSS] Metron Parsers in Nifi

/java/org/apache/nifi/syslog/Syslog5424Reader.java On August 13, 2018 at 09:26:50, Otto Fowler (ottobackwa...@gmail.com) wrote: If we can do the record readers ourselves ( with the parsers inside them ) we can handle the returns. I’ll be doing the net flow 5 readers once the net flow 5 processor

Re: [DISCUSS] Metron Parsers in Nifi

imon On 9 August 2018 at 16:42, Otto Fowler wrote: > I would say that > > - For each configuration parameter we want to pull in, it should be > explicitly configured through a property as well as through a controller > service that accesses the metron zk > - Transformat

Re: [DISCUSS] Pcap query branch completion

- Job cleanup/TTL Documented at least, or a helper script to help yourself if you are in a situation - Expose the Query filter (vs Fixed) in the UI Follow on - Date range limits on queries I don’t see how this won’t be immediately required. I would do this for minimum viable. - Pcap query

Re: [DISCUSS] Metron Parsers in Nifi

parsers (and don't block specialized NiFi implementations that exploit NiFi's feature set), and lets us get things configured in a relatively consistent manner, without losing features, and hopefully requiring a pretty minimal slice of Metron to be useful. On Thu, Aug 9, 2018 at 10:06 AM Otto Fowler

Re: [DISCUSS] Metron Parsers in Nifi

> NiFi processor parses the data and pushes it straight into the enrichment > > topic, saving us the resources of having multiple parsers in storm > > > > Thanks, > > James > > > > 07.08.2018, 11:29, "Otto Fowler" : > > > Why do w

Re: [DISCUSS] Metron Parsers in Nifi

> On Wed, Aug 8, 2018 at 11:46 PM Otto Fowler wrote: > >> I’m seeing >> >> https://github.com/apache/nifi/blob/master/nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/RecordReader.java#L34 >> being quoted as a reason to NOT build Recor

article on swagger and ambari

Worth checking out. https://community.hortonworks.com/articles/210091/how-to-use-swagger-with-ambari-explore-ambari-rest.html

Re: [DISCUSS] Metron Parsers in Nifi

, not a conflicting one. On Tue, Aug 7, 2018 at 11:50 AM Otto Fowler wrote: > A Metron Processor itself isn’t really necessary. A MetronRecordReader ( > either the megalithic or a reader per format ) would be a good approach. > Then have StellarTransformRecord processor that can do Stellar on _any_

Re: [DISCUSS] Metron Parsers in Nifi

er service and provide the json or whatever as one of our properties). On Tue, Aug 7, 2018 at 10:12 AM Otto Fowler wrote: > I think this is a good idea. As I mentioned in the other thread I’ve been > doing a lot of work on Nifi recently. > I think the important thing is that what is

Re: [DISCUSS] Metron Parsers in Nifi

I think this is a good idea. As I mentioned in the other thread I’ve been doing a lot of work on Nifi recently. I think the important thing is that what is done should be done the NiFi way, not bolting the Metron composition onto Nifi. Think of it like the Tao of Unix, the parsers and components

Re: [DISCUSS] Batch Profiler

I think the feature branch is a good idea, but what is in the feature branch or feature branches will have to shake out. I agree in concept with what you have in the jira, but I have two points. 1. We will need a break down of introducing Spark to the stack - required version due to HDP

Re: Security Feature Branch?

on full dev, hence the one PR one unit approach. Does that work, or do we want to review on the basis of a series of untestable bits, and then a final working build PR that pulls it together? Simon On 12 July 2018 at 16:00, Otto Fowler wrote: > Our policy in the past on such things is to requ

Re: Security Feature Branch?

A discussion thread on what you have come up with, the choices you made would be warranted as well. On July 12, 2018 at 11:00:47, Otto Fowler (ottobackwa...@gmail.com) wrote: Our policy in the past on such things is to require that they are broken into small reviewable chunks on a feature

Re: Security Feature Branch?

Our policy in the past on such things is to require that they are broken into small reviewable chunks on a feature branch, even if the end to end working version was more ‘usable’. On July 12, 2018 at 10:51:30, Simon Elliston Ball ( si...@simonellistonball.com) wrote: I've been doing some work

Re: Performance comparison between Grok and Java regex

:19, Muhammed Irshad (irshadkt@gmail.com) wrote: Otto Fowler, Thanks for the reply. I saw it uses same Java regex under the hood. I got bit sceptic by seeing this open issue <https://github.com/thekrakken/java-grok/issues/75> in java-grok which says grok is much slower when compared wit

Re: Performance comparison between Grok and Java regex

Java-Grok IS java regex. It is just a DSL over Java regex. It takes grok expressions ( that can reference other expressions and be compound ) and parses/resolves them and then builds one big regex out of them. Also, Groks, once parsed / used are re-used, so at that point they are like compiled

Re: [DISCUSS] Merging Solr feature branch (METRON-1416) into master

uple +1's on the PR, it's > essentially voting anyway, but this is pretty new in terms of process. > > > > On Fri, Jun 22, 2018 at 12:53 PM Otto Fowler > wrote: > >> If all the PR’s are on master->feature branch. Why do we need testing? >> this is almost a vote sit

Re: Writing enrichment data directly from NiFi with PutHBaseJSON

) wrote: I agree with Simon here, the benefit of providing NiFi tooling is to enable NiFi to use our infrastructure (e.g. our parsers, MaaS, stellar enrichments, etc). This would tie it to Metron pretty closely. On Tue, Jun 5, 2018 at 3:12 PM Otto Fowler wrote: > Nifi releases more of

Re: Writing enrichment data directly from NiFi with PutHBaseJSON

that does increase our release and test burden. On 5 June 2018 at 10:55, Otto Fowler wrote: > Similar to Bro, we may need to release out of cycle. > > > > On June 5, 2018 at 13:17:55, Simon Elliston Ball ( > si...@simonellistonball.com) wrote: > > Do you mean in the se

Re: Writing enrichment data directly from NiFi with PutHBaseJSON

Similar to Bro, we may need to release out of cycle. On June 5, 2018 at 13:17:55, Simon Elliston Ball ( si...@simonellistonball.com) wrote: Do you mean in the sense of a separate module, or are you suggesting we go as far as a sub-project? On 5 June 2018 at 10:08, Otto Fowler wrote: > If

Re: Writing enrichment data directly from NiFi with PutHBaseJSON

to all FlowFile attributes outputting the resulting stellar variable space to either attributes or as json in the content. Is it worth us creating an nifi-metron-bundle. Happy to kick that off, since I'm half way there. Simon On 5 June 2018 at 08:41, Otto Fowler wrote: > We have jiras ab

Re: [DISCUSS] Field conversions

(eol) will need to not the bullet with ES compatibility as some point. Simon > On 5 Jun 2018, at 17:17, Otto Fowler wrote: > > Are there consequences with Kibana as well? queries, visualizations, > templates they may have? > > > On June 5, 2018 at 12:03:44, Nick Allen (n...

Re: [DISCUSS] Field conversions

Are there consequences with Kibana as well? queries, visualizations, templates they may have? On June 5, 2018 at 12:03:44, Nick Allen (n...@nickallen.org) wrote: I just don't know if telling users to do a bulk upgrade of their indices is sufficient enough of an upgrade path. I would expect

Re: Writing enrichment data directly from NiFi with PutHBaseJSON

PutMetronEnrichementRecords* ;) On June 5, 2018 at 10:32:43, Simon Elliston Ball ( si...@simonellistonball.com) wrote: Do we, the community, think it would be a good idea to create a PutMetronEnrichment NiFi processor for this use case? It seems a number of people want to use NiFi to manage

Re: [DISCUSS] Field conversions

It is still our user list and dev list that will have the burden of talking folks through that. On June 5, 2018 at 09:58:32, Casey Stella (ceste...@gmail.com) wrote: To be clear, I'm not even suggesting that we create any tooling here. I'd say just a reference to the ES docs and a call-out in

Re: [VOTE] Metron Release Candidate 0.5.0-RC2

+1 binding Ran the script Validated ui + data in ambari/storm/config ui On May 31, 2018 at 14:35:20, Justin Leet (justinjl...@gmail.com) wrote: This includes a couple fixes from master, in particular two issues that were problematic, METRON-1586

Re: [VOTE] Metron Release Candidate 0.5.0-RC1

ote: >> >> > I'm going to go ahead and cancel RC1, since METRON-1544 looks pretty >> set. >> > >> > A new release candidate will be cut. >> > >> > Results (including my own vote): >> > +1 >> > Nick Allen >> > >&g

Re: [DISCUSS] Refactoring

to be cosmetic refactoring solely due to >> readability concerns. >> >> Just my $0.02 >> >> On Tue, May 29, 2018 at 7:40 PM Otto Fowler >> wrote: >> >>> On top of this, refactoring under another PR’s goals tends to be less >>> doc

Re: [DISCUSS] Refactoring

On top of this, refactoring under another PR’s goals tends to be less documented as to the intent and effect. +1 for the idea, we should have a vote round or edit round on the doc’s specific text. Although I will say, that some things it doesn’t matter how much you break them up wrt reviews. We

Re: [VOTE] Metron Release Candidate 0.5.0-RC1

se since there were two new commits, but I > don't think it was included in this round. > > Jon > > On Sat, May 26, 2018, 10:22 Otto Fowler wrote: > > > Is there a BRO RC # for this? > > > > > > On May 25, 2018 at 14:53:25, Nick Allen (n...@nickallen.org) w

Re: [VOTE] Metron Release Candidate 0.5.0-RC1

was included in this round. Jon On Sat, May 26, 2018, 10:22 Otto Fowler <ottobackwa...@gmail.com> wrote: > Is there a BRO RC # for this? > > > On May 25, 2018 at 14:53:25, Nick Allen (n...@nickallen.org) wrote: > > +1 Release this package as Apache Metron 0.5.0-RC1 > &

Re: [DISCUSS] parser ES + Solr schema abstraction

the process would be config change triggering schema inference triggering diff to old schema optionally triggering a net new version. Does they make sense? Simon On 22 May 2018, at 19:33, Otto Fowler <ottobackwa...@gmail.com> wrote: I’ve also talked with J. Zeolla conceptually storing data i

Re: [DISCUSS] parser ES + Solr schema abstraction

fields) but not to others (removing or reordering fields). This can be resolved by sensible versioning and history aware schema generation. Simon On 22 May 2018 at 15:23, Otto Fowler <ottobackwa...@gmail.com> wrote: > Yes Simon, when I say ‘whatever we would call the complete par

Re: Request for Comment on new Syslog 5424 Parsing library

I am open to adding new syslog parsers or parser ‘specifications’ as I have termed them in. Possibly using grok in the background. On May 21, 2018 at 07:03:40, Otto Fowler (ottobackwa...@gmail.com) wrote: Thanks Ahmed. At the moment, I’m only concerned with RFC 5424 formatted syslog <ht

Request for Comment on new Syslog 5424 Parsing library

There have been some issues and talk about they way we parse syslog, and the deficiencies of our grok and regex based approaches, mainly not supporting structured data as I recall. I played around with it some and decided to try to write an Antlr grammar based on the RFC 5424 spec BNF to parse

Re: [DISCUSS] Metron release 0.5.0

Solr IMO. > > On Wed, May 16, 2018, 7:01 AM Otto Fowler <ottobackwa...@gmail.com> wrote: > > > My question is: Is updating the version a .4->.5 worthy change or would > > adding Solr be that change? > > Should we do another, last .4.x release and bump to .5 when

Re: [DISCUSS] Metron release 0.5.0

My question is: Is updating the version a .4->.5 worthy change or would adding Solr be that change? Should we do another, last .4.x release and bump to .5 when solr hits? On May 15, 2018 at 17:31:27, Nick Allen (n...@nickallen.org) wrote: +1 That plan works for me. IMHO, I don't think there

Re: [DISCUSS] Release?

archIntegrationTest refactor >> > (merrimanr) >> > > > > >> closes apache/metron#909 >> > > > > >> 3 months ago METRON-1426: SensorIndexingConfigController >> > > > IntegrationTest >> > > > > >

Re: [DISCUSS] Release?

Can you run the issues included script and post that for us to see? On May 9, 2018 at 11:14:11, Casey Stella (ceste...@gmail.com) wrote: Is it about time for a release? I know we got some substantial performance changes in since the last release. I think we might have a justification for a

Re: [DISCUSS] Pcap panel architecture

;yarn.timeline-service.enabled" in Ambari to false and then I > > get > > this error: > > > > Unable to parse > > '/hdp/apps/${hdp.version}/mapreduce/mapreduce.tar.gz#mr-framework' as a > > URI, check the setting for mapreduce.application.framework.path >

Re: [DISCUSS] Pcap panel architecture

be a client to that service, the specializes the service operation for the way we want pcap to work. We can then re-use the generic service for other long running yarn things….. On May 7, 2018 at 09:56:51, Otto Fowler (ottobackwa...@gmail.com) wrote: RE: Tracking v. users The submittal and tracking can

Re: [DISCUSS] Pcap panel architecture

ies as a special module? That seems like a very attractive option to me. On Fri, May 4, 2018 at 8:39 AM, Otto Fowler <ottobackwa...@gmail.com> wrote: > From my response on the other thread, but applicable to the backend stuff: > > "The PCAP Query seems more like PCAP Report to

Re: [DISCUSS] Pcap UI user requirements

to investigate some case where the user want to see the whole packet (all the bits and bytes). Like in wireshark, something interactive no? 2018-05-04 14:33 GMT+01:00 Otto Fowler <ottobackwa...@gmail.com>: > The PCAP Query seems more like PCAP Report to me. You are generating a >

Re: [DISCUSS] Pcap panel architecture

>From my response on the other thread, but applicable to the backend stuff: "The PCAP Query seems more like PCAP Report to me. You are generating a report based on parameters. That report is something that takes some time and external process to generate… ie you have to wait for it. I can

Re: [DISCUSS] Pcap UI user requirements

The PCAP Query seems more like PCAP Report to me. You are generating a report based on parameters. That report is something that takes some time and external process to generate… ie you have to wait for it. I can almost imagine a flow where you: * Are in the AlertUI * Ask to generate a PCAP

Re: [DISCUSS] Pcap panel architecture

g position on this other than 1) management is a different feature set from drilling into threat intel, yet many apps still have their management UI combined with the end user experience and 2) we should probably consider pcap in context of a workflow with alerts. On Thu, May 3, 2018 at 4:19 PM, Otto Fowler &

Re: [DISCUSS] Pcap panel architecture

either way really. I'm just not excited about all the MPack code we have to write for a new component. Maybe it won't be that bad. On Thu, May 3, 2018 at 2:50 PM, Otto Fowler <ottobackwa...@gmail.com> wrote: > First thought is why the Alerts-UI and Not a dedicated Query UI? > > >

Re: [DISCUSS] Pcap panel architecture

First thought is why the Alerts-UI and Not a dedicated Query UI? On May 3, 2018 at 14:36:04, Ryan Merriman (merrim...@gmail.com) wrote: We are planning on adding the pcap query feature to the Alerts UI. Before we start this work, I think it is important to get community buy in on the

Re: [VOTE] Development Guidelines Addendum on Inactive Pull Requests

+1 On April 20, 2018 at 09:30:30, Nick Allen (n...@nickallen.org) wrote: I am proposing the following addition to the project's development guidelines [1]. Based on these guidelines, an abandoned pull request can be closed in roughly 6 weeks time (4 weeks of inactivity plus 2 weeks to respond

Re: [DISCUSS] Metron RPM spec changelog

ting, > but > > we can find a way to make that work. The other approach would mean just > > doing a git log on the spec file and grabbing the delta since last > release. > > Side note, I kind of like the idea of having the Jira ticket number in > the > > comment like tha

Re: [DISCUSS] Metron RPM spec changelog

ind a way to make that work. The other approach would mean just > doing a git log on the spec file and grabbing the delta since last release. > Side note, I kind of like the idea of having the Jira ticket number in the > comment like that in the second example. What do you guys think? >

Re: [DISCUSS] Metron RPM spec changelog

I think having the spec file updated with the changes per release is fine, but is the release manager going to do that? If so then the docs need to be updated. Also, we *should* true up any missing entries from the file now. On April 18, 2018 at 11:02:35, Casey Stella (ceste...@gmail.com)

Re: [DISCUSS] Inactive PRs

thing, instead of "submitter", I'll stick with "contributor" because I use that everywhere else. A pull request is 'inactive' if no comments or updates have been made by the contributor in the previous 6 weeks. On Fri, Apr 13, 2018 at 3:06 PM, Otto Fowler <ottobackwa...@gmail

Re: [DISCUSS] Inactive PRs

I would be more explicit that the inactivity was the inactivity of the submitter. It should be clear that this is not for PRs that have not been reviewed, or PRs where the submitter has asked a question or answered a question and the reviewers have abandoned the effort. Not that that ever

java-grok awakening

I have been in contact with the maintainer of java-grok about the status of the project and I am happy to say that there has been activity today, as well as some steps to move it forward and pull some forks back in. https://groups.google.com/forum/#!forum/java-grokhas been created to discuss

Another intermittant build failure?

I had a PR build fail with an issue with the Zookeeper cache. https://travis-ci.org/apache/metron/builds/365122993 Failed tests: ZKConfigurationsCacheIntegrationTest.validateUpdate:230->lambda$validateUpdate$9:230 expected:<{hdfs={index=yaf, batchSize=1, enabled=true},

Re: Unable to build new branch off of master

1.0.1. It does work if you skip that profile; -PHDP-2.5.0.0. I am working on a fix. On Tue, Apr 10, 2018 at 11:38 AM, Otto Fowler <ottobackwa...@gmail.com> wrote: > I’ve create a new branch today, off of current master ( I did fetch apache > and branched off of apache/master ). > I

Unable to build new branch off of master

I’ve create a new branch today, off of current master ( I did fetch apache and branched off of apache/master ). I can’t complete the build portion of vagrant up, even through I have built from the command line fine. After some poking about, I decided to build using the same command as vagrant

Library for docker with junit

Maybe we should look at https://www.testcontainers.org?

Re: [DISCUSS] Generic Syslog Parsing capability for parsers

On 20 Mar 2018, at 21:47, Otto Fowler <ottobackwa...@gmail.com> wrote: > > I entered METRON–1453 <https://issues.apache.org/jira/browse/METRON-1453> a > little while ago while working on the PR#579 > <https://github.com/apache/metron/pull/579>. > > "

[DISCUSS] Generic Syslog Parsing capability for parsers

I entered METRON–1453 a little while ago while working on the PR#579 . "We have several parsers now, with many imaginable that are based on syslog, where the format is SYSLOG HEADER MESSAGE. With

Re: [DISCUSS] Time to remove github updates from dev?

MC member must create the lists using the self-management potal: > > > selfserve.apache.org > > > Once this is done someone can update the INFRA-15988 ticket and the folks > will execute the changes. > > > > On Wed, Jan 31, 2018 at 12:15 AM, Otto Fowler <ottoba

Re: [DISCUSS] Alternatives to split/join enrichment

Also, how are we to measure the effect? Not to get all six sigma ;) On February 22, 2018 at 11:48:41, Otto Fowler (ottobackwa...@gmail.com) wrote: This sounds worth exploring. A couple of questions: * how does this effect the distribution of work through the cluster, and resiliency

Re: [DISCUSS] Alternatives to split/join enrichment

This sounds worth exploring. A couple of questions: * how does this effect the distribution of work through the cluster, and resiliency of the topologies? * Is anyone else doing it like this? * Can we have multiple thread pools and group tasks together ( or separate them ) wrt hbase? On

Re: [DISCUSS] community view/roadmap of threat intel

to the record reader based apis. That should be fine at the O(100s gigabytes) scale in NiFi. Does anyone have any use cases that would still seem like they’d be in the terabytes / existing bulk map reduce approach end? Simon > On 19 Feb 2018, at 14:26, Otto Fowler <ottobackwa...@gmail.com&

Re: [DISCUSS] community view/roadmap of threat intel

There are a couple of use cases here for getting the data. When you _can_ or want to ingest and duplicate the external store 1. Bulk Loading ( from a clean empty state ) 2. Tailing the feed afterwards When you can’t 3. Calling the api ( most likely web ) for reputation or some other thing

Re: ES mpack to include more ES 5 stack properties

I don’t think there are right now. I would recommend entering jira issues for what you haven in mind On February 19, 2018 at 01:02:32, Ali Nazemian (alinazem...@gmail.com) wrote: Hi All, Is there any plan to include more ES 5+ specific properties to Metron mpack? For example, if we want to

Re: [DISCUSS] Profiler Enhancement

lex discussion all by itself, but is something that I would rather handle as a separate effort. On Fri, Feb 2, 2018 at 5:42 PM, Otto Fowler <ottobackwa...@gmail.com> wrote: > You know, I am going to back this up. > I usually thing of replay as replay, profiler or not, but that is not

Re: [DISCUSS] Profiler Enhancement

(n...@nickallen.org) wrote: I think that is definitely a reasonable extension. In this case would we need any additional actions to indicate that data will be overwritten? I am trying to think of other additional needs that this use case has over the others. On Feb 2, 2018 12:38 PM, "Otto F

Re: [DISCUSS] Profiler Enhancement

Scenario 3: As a Security ? I have modified a profile or parser configuration ( replay is replay ), and I want to run the new version against my old data. On February 2, 2018 at 12:19:54, Nick Allen (n...@nickallen.org) wrote: I have been thinking about an enhancement to the Profiler for

Re: [DISCUSS] Persistence store for user profile settings

and groups, and instead of adding an RDBMS, using group prefs and user prefs in the existing KV store (HBase) to reduce the operational maintenance burden on the platform. Simon On 2 Feb 2018, at 12:50, Otto Fowler <ottobackwa...@gmail.com> wrote: It is not uncommon to want to have ‘shared’ prefe

<    1   2   3   4   5   6   >