[GitHub] metron issue #840: METRON-939: Upgrade ElasticSearch and Kibana

Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/840 For reference, I've added some PDF snapshots of the dashboards to the Jira - https://issues.apache.org/jira/secure/attachment/12899952/Metron-Dashboard%20-%20Kibana.pdf -

[GitHub] metron issue #840: METRON-939: Upgrade ElasticSearch and Kibana

Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/840 Ok, the Metron error dashboard is in now. I'll add some additional testing instructions tomorrow, but this PR should be ready for some more vigorous testing. The most easily accessible e2e place to

[GitHub] metron issue #814: METRON-1277 Add match statement to Stellar language

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/814 @cestella any last words before I pull the trigger on this? Any comment? ---

[GitHub] metron pull request #823: METRON-1286 Add MIN & MAX Stellar functions

Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/823#discussion_r153968584 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/OrdinalFunctions.java --- @@ -18,76 +18,91 @@

[GitHub] metron pull request #823: METRON-1286 Add MIN & MAX Stellar functions

Github user jasper-k commented on a diff in the pull request: https://github.com/apache/metron/pull/823#discussion_r153942197 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/OrdinalFunctions.java --- @@ -18,76 +18,91 @@

[GitHub] metron issue #814: METRON-1277 Add match statement to Stellar language

Github user jjmeyer0 commented on the issue: https://github.com/apache/metron/pull/814 once travis build completes successfully this looks good +1. ---

[GitHub] metron issue #814: METRON-1277 Add match statement to Stellar language

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/814 Ok, please have a look at the doc. I'll not that we have not added any notes previously for this type of thing. Also: [METRON-1339: Stellar shellShould have a way to validate deployed

[GitHub] metron pull request #823: METRON-1286 Add MIN & MAX Stellar functions

Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/823#discussion_r153926767 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/OrdinalFunctions.java --- @@ -18,76 +18,91 @@

[GitHub] metron pull request #823: METRON-1286 Add MIN & MAX Stellar functions

Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/823#discussion_r153927131 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/OrdinalFunctions.java --- @@ -18,76 +18,91 @@

[GitHub] metron issue #823: METRON-1286 Add MIN & MAX Stellar functions

Github user cestella commented on the issue: https://github.com/apache/metron/pull/823 Slightly off topic (and not to dissuade from this good work which is much clearer), but as a bit of stellar code golf, I did find out that you can do this with `REDUCE`. For Numerical

[GitHub] metron pull request #823: METRON-1286 Add MIN & MAX Stellar functions

Github user justinleet commented on a diff in the pull request: https://github.com/apache/metron/pull/823#discussion_r153919356 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/OrdinalFunctions.java --- @@ -0,0 +1,93 @@ +/** +

[GitHub] metron pull request #823: METRON-1286 Add MIN & MAX Stellar functions

Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/823#discussion_r153913510 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/OrdinalFunctions.java --- @@ -0,0 +1,93 @@ +/** + *

[GitHub] metron pull request #823: METRON-1286 Add MIN & MAX Stellar functions

Github user justinleet commented on a diff in the pull request: https://github.com/apache/metron/pull/823#discussion_r153899916 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/OrdinalFunctions.java --- @@ -0,0 +1,93 @@ +/** +

[GitHub] metron pull request #823: METRON-1286 Add MIN & MAX Stellar functions

Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/823#discussion_r153898086 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/OrdinalFunctions.java --- @@ -0,0 +1,93 @@ +/** + *

Re: [DISCUSS] e2e test infrastructure

We will also need an ES and Metron REST container for the e2e tests, but yeah you get the idea. I think having the tests be responsible for setup will work. Maybe the next step is to build a simple example and let everyone try it out. If we don't like it, we can throw it away. On Wed, Nov 29,

Re: [DISCUSS] e2e test infrastructure

So we will just have a : ZK container Kafka Container HDFS Container and not deploy any metron stuff to them in the docker setup, the test itself will deploy what it needs and cleanup? On November 29, 2017 at 11:53:46, Ryan Merriman (merrim...@gmail.com) wrote: “I would feel better using

[GitHub] metron pull request #855: METRON-1338 Excluding retry files from RAT check

GitHub user nickwallen opened a pull request: https://github.com/apache/metron/pull/855 METRON-1338 Excluding retry files from RAT check When Vagrant fails, it generates a *.retry file. These files are ignored by Git, but will unnecessarily fail a build because of the Rat check.

[GitHub] metron pull request #854: Experimental Improvements - Feedback Only - Do Not...

GitHub user nickwallen opened a pull request: https://github.com/apache/metron/pull/854 Experimental Improvements - Feedback Only - Do Not Merge This is a PR to get feedback on some minor improvements that I was experimenting with. If the community finds any of this useful, I will

[GitHub] metron issue #814: METRON-1277 Add match statement to Stellar language

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/814 Ok, I'll update the upgrading doc. ---

[GitHub] metron issue #852: METRON-1239 Drop extra dev environments

Github user cestella commented on the issue: https://github.com/apache/metron/pull/852 +1 by inspection. Great job, @nickwallen ---

[GitHub] metron issue #814: METRON-1277 Add match statement to Stellar language

Github user cestella commented on the issue: https://github.com/apache/metron/pull/814 @jjmeyer0 @ottobackwards Yeah, it's definitely possible to break expressions that include variables named `match` and `default`, but that's the danger that I think we have to take for expanding the

Re: [DISCUSS] e2e test infrastructure

“I would feel better using docker if each docker container only had the base services, and did not require a separate but parallel deployment path to ambari” This exactly how it works. There is a container for each base service, just like we now have an in-memory component for each base

Re: [DISCUSS] e2e test infrastructure

Honestly, I'm ok with either the in-memory component approach or the docker approach as long as: - It runs in travis - The infrastructure components are spun up in a way that isolates their classpath - The UI e2e test and the integration tests both use the same infrastructure I

Re: [DISCUSS] e2e test infrastructure

Mike, I think we are in agreement: any solution involving in-memory components would have them running in separate processes vs. a single process like they do now. > On Nov 29, 2017, at 9:14 AM, Michael Miklavcic > wrote: > > I understood the item on "in-memory

[GitHub] metron issue #852: METRON-1239 Drop extra dev environments

Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/852 No problem @ottobackwards . I just deleted the ReadMeUtils and README.vm files in the latest commit. I assume that whatever approach we take for documenting, we want to get rid of these files.

[GitHub] metron issue #852: METRON-1239 Drop extra dev environments

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/852 agreed, didn't meet to hijack. my +1 stands ---

[GitHub] metron issue #814: METRON-1277 Add match statement to Stellar language

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/814 Maybe we need a tool that tests stellar scripts or configurations that the user can run and report? And we can maintain that per release? ---

[GitHub] metron issue #852: METRON-1239 Drop extra dev environments

Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/852 We can tackle the issue of documenting in a discuss thread. I'm not going to tackle that issue on this PR. ---

[GitHub] metron issue #814: METRON-1277 Add match statement to Stellar language

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/814 Great! I am not sure on that. @cestella what do you think? ---

[GitHub] metron issue #852: METRON-1239 Drop extra dev environments

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/852 The api should be documented without requiring the installation of the product. ---

Re: [DISCUSS] e2e test infrastructure

So the issue with metron-docker is that it is all custom setup for metron components, and understanding how to maintain it when you make changes to the system is difficult for the developers. This is a particular issue to me, because I would have to re-write a big chunk of it to accommodate 777.

Re: [DISCUSS] e2e test infrastructure

I understood the item on "in-memory components" to be similar to what we're currently doing in the integration tests, which we cannot and should not do. They are spun up in a single jvm process which causes major problems with classpath isolation. My main point here is to be sure each component is

Re: [DISCUSS] e2e test infrastructure

Thanks for the feedback so far everyone. All good points. Otto, if we did decide to go down the Docker route, we could use /master/metron-contrib/metron-docker as a starting point. The reason I initially create that module was to support Management UI testing because full dev was unusable for

[GitHub] metron issue #814: METRON-1277 Add match statement to Stellar language

Github user jjmeyer0 commented on the issue: https://github.com/apache/metron/pull/814 @ottobackwards this looks good to me now. I re-built everything, ran the tests, and played around in stellar shell. The only question I'm still not sure about is the one that deals with backwards

[GitHub] metron issue #852: METRON-1239 Drop extra dev environments

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/852 i'm not saying keep *this* utility ---

[GitHub] metron issue #852: METRON-1239 Drop extra dev environments

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/852 no, i mean auto documentation of the rest stuff ( and stellar stuff ) from maven builds is a good idea™ ---

Re: [DISCUSS] e2e test infrastructure

I'd also be strongly in favor of having 1 approach to running our e2e and integration tests. On Wed, Nov 29, 2017 at 5:59 AM, Justin Leet wrote: > As an additional consideration, it would be really nice to get our current > set of integration tests to be able to be run on

[GitHub] metron issue #852: METRON-1239 Drop extra dev environments

Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/852 > The issue is that we should use this type of operation, but do it as part of maven no? It doesn't seem that the utility is useful enough to maintain anymore. It had value once, but

[GitHub] metron issue #852: METRON-1239 Drop extra dev environments

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/852 The issue is that we should use this type of operation, but do it as part of maven no? ---

[GitHub] metron issue #852: METRON-1239 Drop extra dev environments

Github user justinleet commented on the issue: https://github.com/apache/metron/pull/852 Looks like @merrimanr commented on the email thread, so I'll copy it here for posterity. > I wrote the ReadMeUtils class a long time ago as a way to make documenting the REST

Re: [GitHub] metron issue #852: METRON-1239 Drop extra dev environments

I wrote the ReadMeUtils class a long time ago as a way to make documenting the REST endpoints easier. The Controller class methods are annotated so that endpoint documentation is displayed in Swagger but it is also duplicated in the README. It seemed like a good idea at the time to provide a

[GitHub] metron issue #852: METRON-1239 Drop extra dev environments

Github user justinleet commented on the issue: https://github.com/apache/metron/pull/852 Glancing briefly, it looks like `ReadMeUtils` uses it as a template for the metron-rest README.md. Just running the main in there overwrites the metron-rest README.md. Which seems very odd,

Re: [DISCUSS] e2e test infrastructure

As an additional consideration, it would be really nice to get our current set of integration tests to be able to be run on this infrastructure as well. Or at least able to be converted in a known manner. Eventually, we could probably split out the integration tests from the unit tests entirely.

[GitHub] metron issue #852: METRON-1239 Drop extra dev environments

Github user anandsubbu commented on the issue: https://github.com/apache/metron/pull/852 > I was confused about that README.vm file. Any idea what that is for? I am not entirely sure, @nickwallen . I could find a lot of similarities between this README.vm and the main Metron

[GitHub] metron issue #852: METRON-1239 Drop extra dev environments

Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/852 > @anandsubbu: I found one more reference to quick dev - here. I was confused about that README.vm file. Any idea what that is for? > @anandsubbu: Also, I could see that the