[GitHub] metron pull request #900: METRON-1411: Fix sed command in Upgrading.md

GitHub user justinleet opened a pull request: https://github.com/apache/metron/pull/900 METRON-1411: Fix sed command in Upgrading.md ## Contributor Comments The sed commands in Upgrading.md for the alert field can be problematic on some versions of sed, including what's on

[GitHub] metron issue #891: METRON-1282 creating a new topic using the rest end point...

Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/891 I think a separate PR is fine. Can you at least move it to a separate method? ---

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user cestella commented on the issue: https://github.com/apache/metron/pull/687 So, I was wrong when I initially looked at this PR, this adds variable updates to the resolver. Should we perhaps separate parsing assignments (e.g. move `StellarAssignment.from` into the language

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user mattf-horton commented on the issue: https://github.com/apache/metron/pull/687 @ottobackwards And yet this discussion primarily relates to whether the assignment operation as defined in Enrichment should be unified with the assignment operation being added to core

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/687 also @mattf-horton, those other things, to be generally useful and not restrictive and assumptive of the host's use cases would have to be generalized _so_ much, I think from an api pov would

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/687 As far as `hosting`. ---

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/687 So, I would say then that we restrict to ONLY core at the moment. And then we start creating interfaces and other api surfaces ( most likely extracted from what we have ) as a follow on.

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user mattf-horton commented on the issue: https://github.com/apache/metron/pull/687 @ottobackwards , that's inarguably true. We can define the language to be just the Stellar Core. We could also define it to include at least some of those very useful other things, which

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/687 the details of how we configure transforms and enrichments, and organize the stellar statements etc, are not required to implement or host stellar. ---

[GitHub] metron issue #899: METRON-1405: Add Boyer-Moore majority vote algorithm to S...

Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/899 Pulling this PR until I can resolve and underlying issue with the counting semantics, however I believe this may by theoretically impossible without a 2-pass model. ---

[GitHub] metron pull request #899: METRON-1405: Add Boyer-Moore majority vote algorit...

Github user mmiklavc closed the pull request at: https://github.com/apache/metron/pull/899 ---

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user mattf-horton commented on the issue: https://github.com/apache/metron/pull/687 > I think we need to keep everything in it's separate pile. We have people asking to host in not metron as recently as today. My interpretation is that, in order to enable hosting

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/687 I vote to get rid of `:=` if we want to make it uniform. I'll also straighten the scope stuff out etc. the whole nine yards if we want to do that. I think that might make this a

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user mattf-horton commented on the issue: https://github.com/apache/metron/pull/687 If one insists that ':=' in Enrichments is a Metron add-on not Stellar, that is equivalent to stating that Enrichment doesn't use Stellar but rather a custom DSL derived from Stellar. The xml

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/687 the top level 'scope' being passed in kind of as it is now. ---

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/687 the scope should be managed by the StellarExpression as a separate stack of linked scope nodes and not in the resolvers. ---

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/687 @mattf-horton > As for := vs =, it's not so terrible to have both as exact synonyms. This gives you both your preferred aesthetics and backward compatibility. Not to keep harping

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user cestella commented on the issue: https://github.com/apache/metron/pull/687 @nickwallen Yes, you are absolutely correct. Right now we do not have an approach to take a list of stellar expressions that are assignments, a variable resolver, a function resolver and execute

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/687 > @cestella: No, both the REPL and enrichments use the same method of assignment, the StellarAssignment class. It's just that assignment in stellar is available in multiple places. Ah,

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/687 I am not opposed to changing the resolvers, or continuing work as it comes out of this discussion ---

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/687 @nickwallen pending this discussion yes, it just adds the capability to the language and stellar common, but not to _*Metron*_. As stated in the PR description: > This PR is

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user cestella commented on the issue: https://github.com/apache/metron/pull/687 @nickwallen No, both the REPL and enrichments use the same method of assignment, the `StellarAssignment` class. It's just that assignment in stellar is available in multiple places. ---

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/687 > @ottobackwards : This is opt-in for the resolvers. The MapVariableResolver does NOT support update, so there is no change to Enrichment, Parsers, Profile, and Shell ( beyond what has been

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/687 OK, The others can comment, but I think proper scope would be implemented within the execution of the call stack, not arbitrarily by the resolvers. Each resolver should be a 'scope'

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/687 > @cestella: No, that's not correct. Stellar enrichments can contain :=. Consider the enrichments here in one of our use-cases. Enrichments can either take a map of enrichments Thanks

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user mattf-horton commented on the issue: https://github.com/apache/metron/pull/687 So a bunch more comments came in while I was writing the above. Nick's comment about enrichments using assignment already of course modifies my comment about Profilers, etc. Just making it

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user mattf-horton commented on the issue: https://github.com/apache/metron/pull/687 As a professed language nut, when you mention the word "variable", I think "scope". And it seems to me that by placing the responsibility for variable assignment/update in the

Re: Some more upgrade fallout... Can't restart Metron Indexing

I assigned METRON-1410 to myself. I will take a shot at addressing this in the ambari service code. On January 18, 2018 at 13:03:18, Laurens Vets (laur...@daemon.be) wrote: On 2018-01-18 09:14, Casey Stella wrote: > So, the challenge here is that our install script isn't smart enough > right >

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user cestella commented on the issue: https://github.com/apache/metron/pull/687 @nickwallen No, that's not correct. Stellar enrichments can contain `:=`. Consider the enrichments

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/687 @cestella @nickwallen I have changed stellar assignment to cover the := already. I think it will continue to work. ---

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/687 > @cestella: What about users with existing stellar enrichments using := ... Oh, and one clarification. No users will have production code (like an enrichment) that uses `:=`. That

Re: [DISCUSS] Upgrading Solr

+1 to both the feature branch and user@ announcement. On Thu, Jan 18, 2018 at 2:45 PM, Otto Fowler wrote: > +1 to the feature branch. > > Also, there have been some questions about solr support recently, I think > when the feature branch > is ready you should announce

Re: [DISCUSS] Upgrading Solr

+1 to the feature branch. Also, there have been some questions about solr support recently, I think when the feature branch is ready you should announce it on user@ too, we may get some help from folks looking for this. On January 18, 2018 at 14:26:14, Justin Leet (justinjl...@gmail.com)

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user cestella commented on the issue: https://github.com/apache/metron/pull/687 hah, well @nickwallen, laziness is often a good trait in a developer. Honestly, I am ok with *not* migrating here in favor of more intuitive approaches. I chose `:=` because I needed the

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/687 > @cestella Migration, sadly sometimes, is also part of user experience. :) Very true. That's why I offered up the ["Alternative

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user cestella commented on the issue: https://github.com/apache/metron/pull/687 Yeah, I'm totally cool with going with `=`, but I will point out one snag. What about users with existing stellar enrichments using `:=`, would we provide a migration path beyond saying, "Hey,

Re: [DISCUSS] Upgrading Solr

+1 to a feature branch for this. I like that approach Justin. On Thu, Jan 18, 2018 at 2:26 PM Justin Leet wrote: > Now that we have ES at a modern version, we should consider bringing Solr > to a modern version as well. > > The focus of this work would be to get us in a

[DISCUSS] Upgrading Solr

Now that we have ES at a modern version, we should consider bringing Solr to a modern version as well. The focus of this work would be to get us in a place where Solr is upgraded, along with the related work of building out the Solr functionality to parity with Elasticsearch. The goal would not

[GitHub] metron pull request #894: METRON-1326: Metron deploy with Kerberos fails on ...

Github user asfgit closed the pull request at: https://github.com/apache/metron/pull/894 ---

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/687 I have not looked at the code yet, but wanted to comment on the assignment discussion. IMO focusing on user experience is the way to go. To that end... * We should use `=` because that

[GitHub] metron issue #894: METRON-1326: Metron deploy with Kerberos fails on Ambari ...

Github user anandsubbu commented on the issue: https://github.com/apache/metron/pull/894 +1 thank you @mmiklavc. Spun up a 12-node Centos 7 cluster with the latest fix and validated the following: * Did 'Stop All Services' -> no issues seen with ES and Kibana services

[GitHub] metron issue #894: METRON-1326: Metron deploy with Kerberos fails on Ambari ...

Github user cestella commented on the issue: https://github.com/apache/metron/pull/894 I'm +1 on this at this point, but suggest we wait until @anandsubbu gives his +1. ---

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/687 also, we aren't consistent, in some places ( transformations ) we use the configuration logical structure to imply assignment, in enrichment it is `:=`, but those are configuration mechanisms

Re: Some more upgrade fallout... Can't restart Metron Indexing

On 2018-01-18 09:14, Casey Stella wrote: So, the challenge here is that our install script isn't smart enough right now to skip creating tables that are already created. One thing you could do is 1. rename the hbase tables for metron (see

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/687 By punt you mean leave this implementation that doesn't implement update in the resolvers? ---

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user cestella commented on the issue: https://github.com/apache/metron/pull/687 Agreed, this is a POV issue. Though from the user perspective they aren't distinguishing between "this is in the REPL" vs "this is in the enrichment topology", so I think we do need to manage the

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/687 POV issue I think. Stellar has the LANGUAGE ( grammar ), the default implementation, and the 'host' execution mechanics. `:=` is part of the host execution mechanics. The

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user cestella commented on the issue: https://github.com/apache/metron/pull/687 well, `:=` is in the shell and also in the enrichment configuration (see [here](https://github.com/apache/metron/tree/master/metron-platform/metron-enrichment#stellar-enrichment-configuration).

Re: Some more upgrade fallout... Can't restart Metron Indexing

JIRAS On January 18, 2018 at 12:14:11, Casey Stella (ceste...@gmail.com) wrote: So, the challenge here is that our install script isn't smart enough right now to skip creating tables that are already created. One thing you could do is 1. rename the hbase tables for metron (see

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/687 @cestella also: := is in the shell not the language. The mechanics of what is LANG and what is MACHINE is part of the issue. ---

Re: Some more upgrade fallout... Can't restart Metron Indexing

So, the challenge here is that our install script isn't smart enough right now to skip creating tables that are already created. One thing you could do is 1. rename the hbase tables for metron (see https://stackoverflow.com/questions/27966072/how-do-you-rename-a-table-in-hbase ) 2.

Re: Travis for Apache/Metron is in trouble

I made an infra ticket: https://issues.apache.org/jira/browse/INFRA-15865 On Thu, Jan 18, 2018 at 11:42 AM, Otto Fowler wrote: > 24hr long build is blocking up master’s travis build. > Who can nuke it? > > ottO >

[GitHub] metron issue #687: METRON-1090 Add Assignment to Stellar Language

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/687 I just removed the conflicts and changed StellarAssignment to account for this ---