Re: [DISCUSS] Upcoming Release

Matt, I think we are currently on version 0.4.2. If that is the case would the next version be 0.4.3? Ryan On Fri, Nov 17, 2017 at 3:31 PM, Matt Foley wrote: > (With release manager hat on) > > The community has proposed a release of Metron in the near future, > focusing on Meta-alerts runnin

Re: [DISCUSS] Upcoming Release

aseProcess-Step15-Cleanup > ) > > So you’re correct that the version of development builds is currently set to > 0.4.2. After we actually make a release of 0.4.2, we’ll change dev build > to 0.4.3 (regardless of whether we expect the following release to > be 0.4.3 or 0.5.0). >

[DISCUSS] e2e test infrastructure

Currently the e2e tests for our Alerts UI depends on full dev being up and running. This is not a good long term solution because it forces a contributor/reviewer to run the tests manually with full dev running. It would be better if the backend services could be made available to the e2e tests w

Re: [GitHub] metron issue #852: METRON-1239 Drop extra dev environments

I wrote the ReadMeUtils class a long time ago as a way to make documenting the REST endpoints easier. The Controller class methods are annotated so that endpoint documentation is displayed in Swagger but it is also duplicated in the README. It seemed like a good idea at the time to provide a uti

Re: [DISCUSS] e2e test infrastructure

at 3:48 PM, Otto Fowler > > wrote: > > > > > As long as there is not a large chuck of custom deployment that has to > be > > > maintained docker sounds ideal. > > > I would like to understand what it would take to create the docker e2e > > env.

Re: [DISCUSS] e2e test infrastructure

another, and that they can be utilized for both the > e2e and integration tests. > >> On Nov 29, 2017 8:00 AM, "Ryan Merriman" wrote: >> >> Thanks for the feedback so far everyone. All good points. >> >> Otto, if we did decide to go down the Docker

Re: [DISCUSS] e2e test infrastructure

nal equivalent and limited to the in memory components > functionality and usage. I apologize if that is in fact what you are getting > at. > > Then we could move the integrations and e2e to them. > > > >> On November 29, 2017 at 10:00:20, Ryan Merriman (merrim...@g

Re: [DISCUSS] e2e test infrastructure

ov 29, 2017 at 1:28 PM, Otto Fowler wrote: > So we will just have a : > > ZK container > Kafka Container > HDFS Container > > and not deploy any metron stuff to them in the docker setup, the test > itself will deploy what it needs and cleanup? > > > On November 29, 2

[DISCUSS] Integration/e2e test infrastructure requirements

The purpose of this discussion is map out what is required to get the POC started with https://github.com/apache/metron/pull/858 into master. The following features were added in the previously mentioned PR: - Dockerfile for Metron REST - Dockerfile for Metron UIs - Docker Compose applic

Re: [DISCUSS] Integration/e2e test infrastructure requirements

go from there. On Mon, Dec 11, 2017 at 4:10 PM, Ryan Merriman wrote: > The purpose of this discussion is map out what is required to get the POC > started with https://github.com/apache/metron/pull/858 into master. > > The following features were added in the previously

Re: [DISCUSS] Integration/e2e test infrastructure requirements

I'm open to ideas. What do you think the title should be? On Wed, Dec 13, 2017 at 2:13 PM, Otto Fowler wrote: > What is the Master Jira going to be? > > > > On December 13, 2017 at 14:36:50, Ryan Merriman (merrim...@gmail.com) > wrote: > > I am going to start the

Re: [DISCUSS] Integration/e2e test infrastructure requirements

eature branch name? I just want to find it and set a watch > on it ;) > > > On December 13, 2017 at 15:29:00, Ryan Merriman (merrim...@gmail.com) > wrote: > > I'm open to ideas. What do you think the title should be? > > On Wed, Dec 13, 2017 at 2:13 PM, Otto Fowler

Re: [DISCUSS] Integration/e2e test infrastructure requirements

, Dec 13, 2017 at 4:41 PM, Ryan Merriman wrote: > Jira is here: https://issues.apache.org/jira/browse/METRON-1352. I am > starting to create sub-tasks based on the requirements outlined above and > included in that Jira description. > > I am compiling a list of modules that we

[DISCUSS] Overcoming developer inertia when spinning up new environments

I want to revisit the idea of providing an alternative container-based approach (Docker, Kubernetes, etc) to spinning up Metron that is faster and uses less resources (a "Metron light"). This would provide a way for reviewers to more quickly review and test out changes. Full dev with ansible will

[DISCUSS] Persistence store for user profile settings

There is currently a PR up for review that allows a user to configure and save the list of facet fields that appear in the left column of the Alerts UI: https://github.com/apache/metron/pull/853. The REST layer has ORM support which means we can store those in a relational database. However I'm

Re: [DISCUSS] Persistence store for user profile settings

es, but thats usually when the > > datastore > > >> is > > >>> a massive set of tables with complex relationships, at least in my > > >>> experience. > > >>> > > >>> We could also use hbase, which probably wouldn'

[DISCUSS] Development environment for UI work

In anticipation of more UI work starting (Alerts UI specifically), I want to kick off a discussion about how we can best provide a backend for UI developers to code against. I believe our full dev environment will not work for this because: 1. It has become more resource intensive over time an

[DISCUSS] Pcap panel architecture

We are planning on adding the pcap query feature to the Alerts UI. Before we start this work, I think it is important to get community buy in on the architectural approach. There are a couple different options. One option is to leverage the existing metron-api module that exposes pcap queries th

Re: [DISCUSS] Pcap panel architecture

owler wrote: > First thought is why the Alerts-UI and Not a dedicated Query UI? > > > On May 3, 2018 at 14:36:04, Ryan Merriman (merrim...@gmail.com) wrote: > > We are planning on adding the pcap query feature to the Alerts UI. Before > we start this work, I think it is im

Re: [DISCUSS] Pcap panel architecture

rection. > > > > There are a few things to consider if we are going to start moving > > everything into Alerts Ui aren’t there? > > > > It may be a better road to bring it in on it’s own like the alerts ui > > effort, so it can be released with ‘qualifiers’ a

Re: [DISCUSS] Pcap panel architecture

I know, I was running with it :) > On May 3, 2018, at 10:21 PM, Michael Miklavcic > wrote: > > Tabs vs spaces was a Silicon Valley joke, man :-) > >> On Thu, May 3, 2018, 8:42 PM Ryan Merriman wrote: >> >> Mike, >> >> I never said there was any

[DISCUSS] Pcap UI user requirements

Continuing a discussion that started in a discuss thread about exposing Pcap query capabilities in the back end. How should we expose this feature to users? Should it be integrated into the Alerts UI or be separate standalone UI? To summarize the general points made in the other thread: - Ad

Re: [DISCUSS] Pcap panel architecture

I started a separate thread on Pcap UI considerations and user requirements at Otto's request. This should help us keep these two related but separate discussions focused. On Fri, May 4, 2018 at 7:19 AM, Michel Sumbul wrote: > Hello, > > > > (Youhouuu my first reply on this kind of mail chain^^

Re: [DISCUSS] Pcap panel architecture

t; > Maybe we can do : > > template -> query parameters -> script => yarn info > yarn info + query info + alert context + yarn status => report info -> > stored in a user’s ‘report queue’ > report persistence added to report info > metron-rest -> api to monitor

Re: [DISCUSS] Pcap panel architecture

ottobackwa...@gmail.com) wrote: > > RE: Tracking v. users > > The submittal and tracking can associate the submitter with the yarn job > and track that, > regardless of the yarn credentials. > > IE> if all submittals and monitoring are by the same yarn user ( Metron )

Re: [DISCUSS] Pcap panel architecture

n jar" to start the > existing pcap service, per the instructions in metron-api/README.md. > > > On Mon, May 7, 2018 at 3:28 PM, Ryan Merriman wrote: > > > To explore the idea of merging metron-api into metron-rest and running > pcap > > queries inside our REST appl

Re: [DISCUSS] Pcap panel architecture

+1 to having metron-api as it's own service and using a gateway type pattern. On Tue, May 8, 2018 at 8:13 AM, Otto Fowler wrote: > Why not have metron-api as it’s own service and use a ‘gateway’ type > pattern in rest? > > > On May 8, 2018 at 08:45:33, Ryan Merriman (merrim

Re: [DISCUSS] Pcap panel architecture

> > operational complexity, and mpack implications of having another server > > installed and running somewhere on the cluster (also, ssl, kerberos, etc > > etc requirements for that service). > > > > On 8 May 2018 at 14:27, Ryan Merriman wrote: > > > > > +

Re: [DISCUSS] Pcap panel architecture

Miklavcic < michael.miklav...@gmail.com> wrote: > Sweet! That's great news. The pom changes are a lot simpler than I > expected. Very nice. > > On Tue, May 8, 2018 at 4:35 PM, Ryan Merriman wrote: > > > Finally figured it out. Commit is here: > > http

Re: [DISCUSS] Pcap panel architecture

ho do packet munging for a living know. > > Jon > > On Wed, May 9, 2018 at 1:00 PM Ryan Merriman wrote: > > > Now that we are confident we can run submit a MR job from our current > REST > > application, is this the desired approach? Just want to confirm. > > > >

Re: [DISCUSS] Pcap panel architecture

results or cleaning up queries (deleting results) would also be limited to that user. Does this sound reasonable? Do we want to add an admin role that can do everything? Is there anything anyone else wants to discuss with regards to authorization or security in general? On Wed, May 9, 2018 at

Re: [DISCUSS] Pcap panel architecture

27;t believe there is a way to "PAUSE" or "SUSPEND" > jobs. That said, I think GET /api/v1/pcap/stop/ is sufficient for > the job management operations. > > On Wed, May 9, 2018 at 11:00 AM, Ryan Merriman > wrote: > > > Now that we are confident we can run

Re: [DISCUSS] Release Manager

Thanks for all your help Matt. On Thu, May 10, 2018 at 10:53 AM, Michael Miklavcic < michael.miklav...@gmail.com> wrote: > Thanks Matt for doing this for the community. > > Justin Leet as new lord commander of the Night's Watch? Aye, dilly, dilly. > > On Thu, May 10, 2018 at 9:07 AM, Justin Leet

Re: [DISCUSS] Release Manager

being our RM. > > On Thu, May 10, 2018 at 12:06 PM Ryan Merriman > wrote: > > > Thanks for all your help Matt. > > > > On Thu, May 10, 2018 at 10:53 AM, Michael Miklavcic < > > michael.miklav...@gmail.com> wrote: > > > > > Thanks Matt for d

Re: [DISCUSS] Pcap panel architecture

> > On Thu, May 10, 2018 at 10:16 AM Ryan Merriman > wrote: > > > Mike, I believe the /pcapGetter/getPcapsByIdentifiers endpoint exposes > the > > fixed query option which we have covered. I agree with you that > > deprecating the metron-api module should be a

Re: [DISCUSS] Pcap panel architecture

for analysis Jon. > > > > > > On May 11, 2018 at 10:14:02, zeo...@gmail.com (zeo...@gmail.com) wrote: > > > > I think baby steps are fine - admin gets access to all, otherwise you > only > > see your own pcaps, but we file a jira for a future add of API securi

Re: [DISCUSS] Pcap panel architecture

Yes there will be an admin role that can read and delete all. On Fri, May 11, 2018 at 4:11 PM, Otto Fowler wrote: > Do we at least require a admin/super user? See’s all the queues and jobs? > > > On May 11, 2018 at 17:03:34, Ryan Merriman (merrim...@gmail.com) wrote: > > T

[DISCUSS] Field conversions

We've been dealing with a reoccurring challenge in Metron. It is common for various fields to contain '.' characters for the purpose of making them more readable, namespacing, etc. At one point we only supported Elasticsearch 2.3 which did not allow dots and forced us to use ':' instead. This li

Re: [DISCUSS] Field conversions

onversion/upgrade > script to change '.' to ':'? > > > On 2018-06-04 13:55, Ryan Merriman wrote: > >> We've been dealing with a reoccurring challenge in Metron. It is common >> for various fields to contain '.' characters for the purpose o

Re: [DISCUSS] Field conversions

's another way to skin this cat. > > Ultimately, I think we should just ditch this field transformation > business, as Laurens said, as long as we have a utility to transform > existing data. > > On Tue, Jun 5, 2018 at 8:54 AM Ryan Merriman wrote: > > > Having 2 diff

[DISCUSS] Deprecating metron-api

We are currently working on adding pcap query capabilities to the Alerts UI as part of https://issues.apache.org/jira/browse/METRON-1554. This involves exposing pcap endpoints in our REST application which will make metron-api obsolete. Is anyone currently using this module? Are there any object

Re: [DISCUSS] Deprecating metron-api

there. You might want > to cc user@ as well, to get a broader set of input for the "are people > using it?" question. > > On Fri, Jun 29, 2018 at 5:21 PM Ryan Merriman wrote: > > > We are currently working on adding pcap query capabilities to the Alerts > UI > &g

[DISCUSS] Pcap query branch completion

We are nearing a fully functional Pcap query feature branch. I want to take a moment before we merge to review the original discussion threads and make sure the community is happy with the state of this feature branch before we accept it into master. The original discuss threads are located at: -

Re: [DISCUSS] Pcap query branch completion

Thanks for the feedback Otto. I have created a sub task for documenting the Job cleanup documentation: https://issues.apache.org/jira/browse/METRON-1737. I completely agree with you there, this needs to be documented. For the others you marked "Follow on" I will create follow on tasks in Jira.

Re: [DISCUSS] Pcap query branch completion

e offer guidance on storm > tuning, and that is conceptually the same isn’t it? That is why it comes > to mind. > Maybe this can be a follow on, in the tuning guide? > > On August 13, 2018 at 17:36:41, Ryan Merriman (merrim...@gmail.com) wrote: > > - Date range limits on quer

Re: [DISCUSS] Pcap query branch completion

Otto, I believe the items you requested are in the feature branch now. Is there anything outstanding that we missed? The Jiras for the Pcap feature branch should be up to date: https://issues.apache.org/jira/browse/METRON-1554 On Mon, Aug 13, 2018 at 5:13 PM, Ryan Merriman wrote: > - D

Re: [DISCUSS] Pcap query branch completion

se who > > contributed to it! > > > > On Thu, Aug 16, 2018 at 8:27 AM Otto Fowler > wrote: > > > >> Looks good, thanks! > >> > >> On August 15, 2018 at 19:38:12, Ryan Merriman (merrim...@gmail.com) > wrote: > >> > >> O

Re: [DISCUSS] Pcap query branch completion

t; On Thu, Aug 16, 2018, 4:17 PM Ryan Merriman wrote: > > > We discovered a bug in our testing and felt it should be fixed before we > > merge. There is a PR up for review that already has a +1: > > https://github.com/apache/metron/pull/1168. I don't anticipate this >

[DISCUSS] Internal Metron fields

I recently worked on a PR that involved changing the default behavior of the ElasticsearchWriter to store data using field names with the default Metron separator, dots. One of the unfortunate consequences of this is that although dots are allowed in more recent versions of ES, it changes how thes

Re: [DISCUSS] Internal Metron fields

g those keys, which would be eliminated by > refactoring. > >> On Fri, Sep 7, 2018, 3:50 PM Ryan Merriman wrote: >> >> I recently worked on a PR that involved changing the default behavior of >> the ElasticsearchWriter to store data using field names with the

Re: [DISCUSS] Knox SSO feature branch review and features

I have reviewed a couple different PRs so I'll add some context where I can. Obviously Simon would be the most qualified to answer but I'll add my thoughts. For question 1, while they may not all be necessary I think it does make sense to include them in this feature branch if our primary goal is

Re: [DISCUSS] Batch Profiler Feature Branch

+1 from me. Great work. On Thu, Sep 27, 2018 at 12:41 PM Justin Leet wrote: > I'm +1 on merging the feature branch into master. There's a lot of good > work here, and it's definitely been nice to see the couple remaining > improvements make it in. > > Thanks a lot for the contribution, this is

[DISCUSS] Stellar REST client

I want to open up discussion around adding a Stellar REST client function. There are services available to enrich security telemetry and they are commonly exposed through a REST interface. The primary purpose of this discuss thread to collect requirements from the community and agree on a general

Re: [DISCUSS] Stellar REST client

o not do so, I'd suggest we use HTTP > Components as it's another apache project and under active > development/support. I'd also be ok with OkHttp if it's actively > maintained. > > On Fri, Oct 19, 2018 at 11:46 AM Ryan Merriman > wrote: > > > I

Re: [DISCUSS] Stellar REST client

e: > I believe the issue of introducing and supporting higher latency > enrichments is a systemic one, and should be solved as such, > with the rest and other higher latency enrichments build on top of that > framework. > > > > > On October 19, 2018 at 12:22:28, R

Re: [DISCUSS] Deprecate split-join enrichment topology in favor of unified enrichment topology

+1 On Thu, Nov 1, 2018 at 5:38 PM Casey Stella wrote: > +1 > On Thu, Nov 1, 2018 at 18:34 Nick Allen wrote: > > > +1 > > > > On Thu, Nov 1, 2018, 6:27 PM Justin Leet wrote: > > > > > +1, I haven't seen any case where the split-join topology isn't made > > > obsolete by the unified topology. >

[DISCUSS] Adding Apache Knox

I want to open a discussion around adding Apache Knox to our architecture. I believe there are some benefits it offers in a couple different areas: 1. It provides a proxy to endpoints for commonly used Hadoop-ecosystem components that we use. This would simplify our web architecture by a

Re: [DISCUSS] Knox SSO feature branch review and features

t; > > without > > > > >>> saying, but I just want to be doubly sure that we're installing > > the > > > > >>> default > > > > >>> via the standard install mechanism as opposed to something > > separate > > >

Re: [DISCUSS] Knox SSO feature branch review and features

x an optional feature that requires setup > > with > > > the help of some documentation (like Kerberos) while keeping the system > > the > > > way it is now by default. > > > Sounds good to me. > > > > > > > I imagine we'll deprecate JDB

Re: [DISCUSS] Knox SSO feature branch review and features

w of several > organisations who are blocked from using Metron by the presence of the JDBC > authentication method in its current form. > > Simon > > On Mon, 12 Nov 2018 at 14:36, Ryan Merriman wrote: > > > Let me clarify on exposing both legacy and Knox URLs at the same ti

Re: [DISCUSS] Knox SSO feature branch review and features

igrate over to LDAP and then be deprecated. The code should still be > pluggable and if anyone wanted to extend it to work with JDBC they could, > or if people wanted to plug in another proxy they could, but this is not > something we would officially support. > > Thanks, > James

Re: [DISCUSS] Knox SSO feature branch review and features

d others that are app-specific. Some > explanation of what this looks like would be helpful. > 3) Sounds like this follows pretty naturally from #1 > 4) Relates to #2. I think we need some guidance on what a manual vs > MPack/automated install would look like. > > Cheers, >

Re: [DISCUSS] Knox SSO feature branch review and features

ave we looked at how others integrate? Having all the code and being > > able to track stuff is kind of the point of this whole thing isn’t it? > > > > Maybe this is implied and I’m missing it, if so I apologize. > > > > I think consistency with the rest of the hadoop stack with kn

Re: [DISCUSS] Knox SSO feature branch review and features

ng look like for UI testing with Knox enabled? > Anything we lose from a dev testability standpoint? The discussion of > defaults sounds reasonable to me, and I'd like to understand any other > tradeoffs there may be for non-prod deployments like full dev. > > On Fri, Nov 1

Re: [DISCUSS] Knox SSO feature branch review and features

ments from Otto and Simon. > > On Fri, Nov 16, 2018 at 9:49 AM Ryan Merriman wrote: > > > I was still able to spin up the UI locally and debug in my testing. I am > > in complete agreement, we need to ensure the developer experience doesn't > > change. &

Re: [DISCUSS] Knox SSO feature branch review and features

:54 AM Ryan Merriman wrote: > I would also add that defaulting to Knox being on simplifies things at a > technical level. > > On Fri, Nov 16, 2018 at 10:52 AM Michael Miklavcic < > michael.miklav...@gmail.com> wrote: > >> That's fantastic, thanks for that detail

[DISCUSS] Writer class refactor

Recently there was a bug reported by a user where a parser that emits multiple messages from a single tuple doesn't work correctly: https://issues.apache.org/jira/browse/METRON-1968. This has exposed a problem with how the writer classes work. The fundamental issue is this: the writer classes op

Re: [DISCUSS] Writer class refactor

rty writers, and would structure > them as such. > > > > On January 18, 2019 at 13:55:00, Ryan Merriman (merrim...@gmail.com) > wrote: > > Recently there was a bug reported by a user where a parser that emits > multiple messages from a single tuple doesn't work correct

Re: [DISCUSS] Writer class refactor

>> point about not passing 2 separate lists, I suggest that we modify the API >> by passing in something like Map> so that the >> tuples always get acked with respect to their messages. This way we can >> avoid the tuple-message batch boundary problem by ensuring we only ack

[DISCUSS] Architecture documentation

Recently I submitted a PR that introduces a large number of changes to a critical part of our code base. Reviewers feel like it is significant enough to document at an architectural level (and I agree). There are a couple points I would like to clarify.

Re: [DISCUSS] Architecture documentation

of data in Metron. I think we probably want an updated version > of this wiki entry from back in the day - > https://cwiki.apache.org/confluence/display/METRON/Metron+Architecture > > Best, > Mike > > > On Mon, Feb 25, 2019 at 7:18 AM Nick Allen wrote: > >

[DISCUSS] Upgrading HBase and Kafka support

I have been researching the effort involved to upgrade to HDP 3. Along the way I've found a couple challenging issues that we will need to solve, both involving our integration testing strategy. The first issue is Kafka. We are moving from 0.10.0 to 2.0.0 and there have been significant changes

Re: [DISCUSS] Next Release

Jon is correct. I am actively working on this and hope to have it completed soon. I realize it will hold up the release so it's a priority for me. On Sat, Mar 30, 2019 at 6:09 PM zeo...@gmail.com wrote: > Isn't the documentation already in progress? > > https://github.com/apache/metron/pull/

Re: [VOTE] Metron Release Candidate 0.7.1-RC1

I am working on the backend change mentioned above ( https://issues.apache.org/jira/browse/METRON-2034) and should a PR up today. On Mon, Apr 29, 2019 at 1:16 PM Tamás Fodor wrote: > As Justin pointed out, we've already implemented the frontend related part > of the aggregation in https://github

Re: [DISCUSS] Metron RPM spec file changelog

I vote we get rid of them. It's easy enough to look through the commit history to see what changed and when. If there is a need to explain a change I think an inline comment would be more appropriate. On Thu, May 30, 2019 at 3:52 PM Michael Miklavcic < michael.miklav...@gmail.com> wrote: > Duri

[DISCUSS] Shaded jar classifiers

I recently opened a PR that has potential to significantly change (for the better in my opinion) the way our Maven build process works. I want to highlight this and get any feedback on potential issues that may come with this change. I frequently run i

Re: [DISCUSS] Shaded jar classifiers

u spin it up in full-dev > (to ensure ambari doesn't have any dependencies on the jar names)? > > Other than that, I'm +1 to the effort! > > On Mon, Jun 3, 2019 at 8:55 AM Ryan Merriman wrote: > > > I recently opened a PR <https://github.com/apache/metron/pull/1436

Re: [DISCUSS] Parser Aggregation in Management UI

"We planning to add the changes to the latest PR as additional commits to avoid impacting the PR sequence. We will refer to the source PR in the commit message of the fix. Also adding a link to the comment section of the source PR of the change request to the fixing commit to make them connected."

Re: Discuss: Time to update bundled SOLR support?

aid that, especially for feature branches, having more eyes on it > > is generally pretty helpful if you're interested in hopping in to catch > any > > issues or opportunities. I believe the people most involved are Mike > > Miklavcic, Nick Allen, and Ryan Merriman, so they

Re: [VOTE] Move Apache Metron to the Apache Attic and Dissolve PMC

+1 > On Nov 16, 2020, at 5:19 PM, David Lyle wrote: > > +1 > >> On Mon, Nov 16, 2020 at 3:10 PM Michael Miklavcic < >> michael.miklav...@gmail.com> wrote: >> >> +1 >> >>> On Mon, Nov 16, 2020 at 7:01 AM Justin Leet wrote: >>> >>> Hi all, >>> >>> This is a vote thread to retire Metron to t

Re: [DISCUSS] next release proposal

Matt, I started a discussion around Kerberos support as a prerequisite for MPack work and the consensus was that a service should support Kerberos before it's included in the MPack. There is a PR out there for Kerberos support for REST (https://github.com/apache/incubator-metron/pull/535) but it

Re: [DISCUSS] next release proposal

Jon, I've already started reviewing the pycapa PR but a second pair of eyes never hurts. On Thu, Apr 20, 2017 at 6:15 AM, zeo...@gmail.com wrote: > At a high level this looks good to me. I'm going to try and pick out a PR > that needs some testing and hit it today (pycapa?). I found an issue

Re: Reducing Warnings in Build

I think Nick brings up some good points. Would there ever be a reason to not use UTF8 as the default from parsing a message on? All the tools we use for analytics work with UTF8 (am I wrong?). The only case I can see needing a configurable charset would be if a message coming from a sensor were

Re: metron UI

What features would you like to see included? On Mon, Apr 24, 2017 at 8:51 AM, moshe jarusalem wrote: > Hi All, > I have recently run metron UI. I am a bit surprised because it has very few > features such as configuring some ingestions and topologies. > > Have I not configured it properly or it

[DISCUSS] Regression introduced in Full Dev

A regression was introduced recently that breaks full dev. I've narrowed down the commit that introduced it and have submitted a PR to revert that commit: https://github.com/apache/incubator-metron/pull/549. Given there has been confusion recently over our deployment build process, I think it's

Re: metron UI

and was merged into master not long ago. It's definitely open to > being iterated and improved on, and getting feedback on direction users > would like it to go would be a great contribution in and of itself > (especially as it gets exercised and people want new features or > imp

Re: [DISCUSS] Metron Rest to Install Parser (METRON-258)

I think leveraging the REST application would work for this use case. Services already exist for most of the functions listed in your pseudocode (HDFS read/write, Zookeeper read/write). Asynchronous functions are also supported so no issues there. On Thu, Apr 27, 2017 at 9:09 AM, Otto Fowler wro

Re: [DISCUSS] Metron Rest to Install Parser (METRON-258)

could look at? > > > On April 27, 2017 at 10:45:09, Otto Fowler (ottobackwa...@gmail.com) > wrote: > > I thought as much. I have not worked with the framework before however. > > > On April 27, 2017 at 10:20:20, Ryan Merriman (merrim...@gmail.com) wrote: > > I think lever

Re: [Discussion] Access to Ambari Variables

You would supply Ambari credentials when you make the Ambari (not ours) REST call. On Mon, May 1, 2017 at 3:14 PM, Otto Fowler wrote: > But we don’t have SSO, so that would required the ambari credentials be > available to the rest api > > > On May 1, 2017 at 16:10:03, Simon Elliston Ball ( > si

Re: [GitHub] incubator-metron issue #562: METRON-915 add node and npm to platform_info.sh

We are using a mvn plugin that automatically installs the correct version of node and npm locally, at least for the management UI. Are there other parts of the project that depend on these tools? Is it desirable to include this even if they aren't a prerequisite for building? I don't think

Re: Error building in Travis after taking master

You might want to try clearing the mvn cache. I've had travis get into a bad state before because of corrupt maven artifacts. On Wed, May 3, 2017 at 10:26 AM, Otto Fowler wrote: > https://travis-ci.org/ottobackwards/incubator-metron/builds/228364894?utm_ > source=email&utm_medium=notification >

Re: Error building in Travis after taking master

om) >>> wrote: >>> >>> Also Otto, are you able to build locally with that branch and same merge >>> with master? >>> >>> On Wed, May 3, 2017 at 9:51 AM, Justin Leet >>> wrote: >>> >>>> I'm also curious

Re: [DISCUSS] REST + ambari

It already works the way Simon describes. On Mon, May 8, 2017 at 8:08 AM, Simon Elliston Ball < si...@simonellistonball.com> wrote: > My proposal would be that the REST api use it’s application.yml for all > the parameters and have settings it needs included in that. E.g. the metron > directory y

Re: Metron REST not starting

Did you install the MySQL client jar? On Mon, May 8, 2017 at 12:40 PM, Laurens Vets wrote: > Hello list, > > I've installed Metron via Ambari and everything works except the Metron > REST interface. It fails with the following error message: "Cannot load > driver class: com.mysql.jdbc.Driver". I

Re: Site-Book: mvn site never completes

Can we adjust the format of that table? Do you know what about the table is breaking it? I used other examples from other READMEs in Metron for that table so it's not a new thing. On Tue, May 9, 2017 at 2:05 PM, Justin Leet wrote: > Looks like the table of environment variables kills it. An i

Re: [MASTER BROKEN] Possible Issue in METRON.SPEC in master

You are correct. I just finished fixing this to get the RPMs to build. Should be an easy fix. On Tue, May 9, 2017 at 3:51 PM, Otto Fowler wrote: > I just tried to merge in master, and got conflicts in metron.spec. Which > is not unusual since spec files are awesome. > But I was sing a head

Re: mvn building errors with 0.3.1

Kevin, I think your error is related to npm. Can you attach the full log file? On Mon, May 15, 2017 at 5:23 PM, Kevin Waterson wrote: > I am getting something similar.. > > [INFO] BUILD FAILURE > [INFO] > > [INFO] Total ti

Re: Trying to spin up Metron in EC2: Failed

That happens when you don't have the zookeeper url configured correctly. Can you check the contents of the /etc/sysconfig/metron file on the Metron host? On Wed, May 17, 2017 at 1:36 PM, Laurens Vets wrote: > For testing purposes, I decided to spin up the default Metron AWS config. > This result

Re: Build failures

Can you confirm you're on the master branch? I see "metron-streaming" in your path to RestTestingUtil and that was changed a LONG time ago. You're likely on a really old branch. Ryan On Wed, Jun 28, 2017 at 3:27 PM, Vasco Yordanov wrote: > Hello , I just forked from github and it seems that "

Re: [VOTE] Apache Metron 0.4.0 release

+1 (binding) - Verified Keys - Verified mvn clean install - Ran full dev and performed several smoke tests including: - tested several REST endpoints - verified data in ES - tested Management UI - verified Storm topologies in Storm UI - verified data in Kiban

  1   2   >