Github user justinleet commented on a diff in the pull request:
https://github.com/apache/metron/pull/702#discussion_r137762973
--- Diff:
metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchDao.java
---
@@ -330,43 +334,112 @@ public
GitHub user merrimanr opened a pull request:
https://github.com/apache/metron/pull/737
METRON-1161: Add ability to edit parser command line options in the
management UI
## Contributor Comments
This PR is a follow up to https://github.com/apache/metron/pull/717 and
exposes the
Github user justinleet commented on the issue:
https://github.com/apache/metron/pull/702
This is really good, thanks for the contribution! Are we intending (as a
follow-on activity), to enable something like top hits aggregation in here so
we can get the documents in the buckets
Github user asfgit closed the pull request at:
https://github.com/apache/metron/pull/730
---
Github user justinleet commented on the issue:
https://github.com/apache/metron/pull/724
@cestella That absolutely makes sense. Thanks a lot for adding it.
+1
---
Github user justinleet commented on the issue:
https://github.com/apache/metron/pull/735
+1 by inspection. Thanks for the contribution
---
GitHub user cestella opened a pull request:
https://github.com/apache/metron/pull/738
METRON-1166: Stellar short circuiting fails when a complex condition using
a boolean op is followed by the opposite boolean op
## Contributor Comments
Rather subtle one here. Stellar short
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/740#discussion_r137826193
--- Diff:
metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/shell/StellarShell.java
---
@@ -308,17 +315,68 @@ private void
Github user cestella commented on a diff in the pull request:
https://github.com/apache/metron/pull/736#discussion_r137808595
--- Diff:
metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/stellar/ProfilerFunctions.java
---
@@ -131,50 +133,99 @@
Github user asfgit closed the pull request at:
https://github.com/apache/metron/pull/729
---
GitHub user ottobackwards opened a pull request:
https://github.com/apache/metron/pull/739
METRON-1165 [FEATURE-BRANCH] Add ability for BundleSystem to add bundles
after initialization
As part of supporting parser extensions and the bundle system hosted in
other applications than
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/736#discussion_r137822891
--- Diff:
metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/stellar/ProfilerFunctions.java
---
@@ -131,50 +133,99
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/736#discussion_r137825517
--- Diff:
metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/stellar/ProfilerFunctions.java
---
@@ -131,50 +133,99
Github user justinleet commented on the issue:
https://github.com/apache/metron/pull/738
+1, pending Travis. Good catch, thanks a lot for the contribution.
---
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/729
+1. The issues I was seeing where a problem in my own code ( surprise
right? ). This fix is correct and appropriate for the scope it seeks to
address. It is really good work.
---
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/739
@mattf-horton
---
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/740#discussion_r137841078
--- Diff:
metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/shell/StellarShell.java
---
@@ -308,17 +315,68 @@ private void
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/740#discussion_r137841201
--- Diff:
metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/shell/StellarShell.java
---
@@ -308,17 +315,68 @@ private void
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/740#discussion_r137841605
--- Diff:
metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/shell/StellarShell.java
---
@@ -308,17 +315,68 @@ private void
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/736
+1 by inspection, lgtm!
---
Github user asfgit closed the pull request at:
https://github.com/apache/metron/pull/738
---
Github user justinleet commented on the issue:
https://github.com/apache/metron/pull/741
@ottobackwards From the ticket description, it looks like you thought there
was a more involved refactoring for doing something like this. Are you good
with this sort of solution (at least as an
GitHub user justinleet opened a pull request:
https://github.com/apache/metron/pull/741
METRON-1153 HDFS HdfsWriter never recovers from exceptions
## Contributor Comments
Added a try-catch around the actual write that will rotate the file and try
again if there's a stream
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/740
How would we, say, specify a global config value as a map (e.g. foo={'bar'
: 'blah'})? global config is also exposed as variables stellar in the parser
and enrichments, so this could very well be
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/metron/pull/734#discussion_r137874476
--- Diff: metron-interface/metron-rest/README.md ---
@@ -361,6 +363,21 @@ Request and Response objects are JSON formatted. The
JSON schemas are available
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/740#discussion_r137838874
--- Diff:
metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/shell/StellarShell.java
---
@@ -308,17 +315,68 @@ private void
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/740#discussion_r137864851
--- Diff:
metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/shell/StellarShell.java
---
@@ -308,17 +315,68 @@ private void
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/740
I think I'd be ok without that capability if we had an ability to pass in a
global config as a JSON blob, like we do the initial set of variables. What do
you think?
---
Github user justinleet commented on a diff in the pull request:
https://github.com/apache/metron/pull/734#discussion_r137874898
--- Diff: metron-interface/metron-rest/README.md ---
@@ -361,6 +363,21 @@ Request and Response objects are JSON formatted. The
JSON schemas are
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/740#discussion_r137840091
--- Diff:
metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/shell/StellarShell.java
---
@@ -308,17 +315,68 @@ private void
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/741
This looks good to me as a stopgap. I'd like to understand eventually why
this happens, but it appears very intermittent, so this isn't a bad solution at
least as a stopgap.
+1 by
Github user justinleet commented on a diff in the pull request:
https://github.com/apache/metron/pull/742#discussion_r137883199
--- Diff:
metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/StringFunctionsTest.java
---
@@ -408,6 +408,19 @@ public
Github user justinleet commented on the issue:
https://github.com/apache/metron/pull/742
+1, thanks a lot for the contribution. And for catching that we never
actually had this
---
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/metron/pull/734#discussion_r137889738
--- Diff:
metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/integration/ElasticsearchMetaAlertIntegrationTest.java
---
Github user asfgit closed the pull request at:
https://github.com/apache/metron/pull/736
---
Github user cestella commented on a diff in the pull request:
https://github.com/apache/metron/pull/732#discussion_r137895036
--- Diff:
metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/writer/SimpleHbaseEnrichmentWriter.java
---
@@ -134,6 +136,39 @@
Github user merrimanr commented on the issue:
https://github.com/apache/metron/pull/700
Thanks @iraghumitra. I tested this in full dev and everything worked as
expected. +1
---
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/740
so, what happens if we modify a field via `%define` from a stellar shell
started with `-z` ?
---
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/740
Those options seem to add a bit more complexity than I'd like. How about
we just restrict %define to strings only for now. If you need to use complex
types, then you can still do the
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/740
"a bit more complexity than I'd like" may just need to be read as "Nick is
lazy", but I'll let you be the judge
---
Looks like everything’s in now. Thanks very much everyone for the efforts!
I’m spinning the RC and will have it out for vote shortly.
--Matt
On 9/7/17, 11:41 PM, "Matt Foley" wrote:
METRON-1163 turns out to be invalid, due to a bad maven settings file. It
is no
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/740
Yes, I think it would be useful to pass in a global config as a command
line argument. Maybe something like...
```
bin/stellar -g config/zookeeper/global.json
```
I think we
Github user merrimanr commented on the issue:
https://github.com/apache/metron/pull/734
Great job on this. I spent a lot of time testing and had trouble finding
anything wrong. The one thing I did notice was that the "timestamp" field was
missing from documents in the metaalerts
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/740
And in terms of the complex types I can see the use case definitely. In
terms of the REPL and the changes in this PR, a `%define` does the same as when
GLOBAL_CONFIG gets created with a `-z` for
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/metron/pull/702#discussion_r137893211
--- Diff:
metron-interface/metron-rest/src/main/java/org/apache/metron/rest/config/IndexConfig.java
---
@@ -56,8 +52,10 @@ public IndexDao indexDao() {
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/metron/pull/702#discussion_r137893133
--- Diff:
metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/search/GroupOrderType.java
---
@@ -0,0 +1,39 @@
+/**
+ *
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/metron/pull/702#discussion_r137893095
--- Diff:
metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchDao.java
---
@@ -330,43 +334,112 @@ public
Github user asfgit closed the pull request at:
https://github.com/apache/metron/pull/700
---
GitHub user cestella opened a pull request:
https://github.com/apache/metron/pull/742
METRON-1168: Add SUBSTRING method to stellar
## Contributor Comments
Add `SUBSTRING` function to stellar.
Testing:
Open up the REPL and play with `SUBSTRING`
## Pull
Matt,
This was committed a few hours. I think you saw it but just wanted to make sure.
Ryan
> On Sep 7, 2017, at 11:26 AM, Matt Foley wrote:
>
> Okay. Please ping when committed.
> Also, any input on https://issues.apache.org/jira/browse/METRON-1163 ?
>
> On 9/7/17, 7:39
Hi Ryan, yup, got it. It’s in.
On 9/8/17, 1:25 PM, "Ryan Merriman" wrote:
Matt,
This was committed a few hours. I think you saw it but just wanted to make
sure.
Ryan
> On Sep 7, 2017, at 11:26 AM, Matt Foley wrote:
>
Github user merrimanr commented on the issue:
https://github.com/apache/metron/pull/702
The latest commit should address the feedback so far. I don't feel like
the top hits aggregation is necessary at this point but it could be in the
future. I would prefer to wait and keep it
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/740
So, one way to do this is to allow users to refer to stellar fields:
```
my_complex_field := { 'blah' : 7 }
%define nicks_new_global_field=_complex_field
```
Another option
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/740
you could also make `%define nicks_new_global_field := my_complex_field`
That makes somewhat more sense since stellar assignment is `:=` and property
assignment is `=`
---
GitHub user cestella opened a pull request:
https://github.com/apache/metron/pull/744
METRON-1169: dependency checker has not been running in travis
## Contributor Comments
Name says it all
## Pull Request Checklist
Thank you for submitting a contribution
Github user asfgit closed the pull request at:
https://github.com/apache/metron/pull/741
---
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/741#discussion_r137913278
--- Diff:
metron-platform/metron-writer/src/main/java/org/apache/metron/writer/hdfs/SourceHandler.java
---
@@ -64,19 +64,34 @@ public
Github user cestella commented on a diff in the pull request:
https://github.com/apache/metron/pull/742#discussion_r137913436
--- Diff:
metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/StringFunctions.java
---
@@ -321,6 +321,46 @@ public Object
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/741#discussion_r137913406
--- Diff:
metron-platform/metron-writer/src/main/java/org/apache/metron/writer/hdfs/SourceHandler.java
---
@@ -64,19 +64,34 @@ public
Github user cestella commented on a diff in the pull request:
https://github.com/apache/metron/pull/742#discussion_r137913443
--- Diff:
metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/StringFunctions.java
---
@@ -321,6 +321,46 @@ public Object
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/742
I am going to create a follow-on PR to this to address your issues.
---
Github user cestella commented on a diff in the pull request:
https://github.com/apache/metron/pull/742#discussion_r137913498
--- Diff:
metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/StringFunctionsTest.java
---
@@ -408,6 +408,23 @@ public
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/741#discussion_r137913296
--- Diff:
metron-platform/metron-writer/src/main/java/org/apache/metron/writer/hdfs/SourceHandler.java
---
@@ -64,19 +64,34 @@ public
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/741
@justinleet My question with regards to the necessity for a refactor came
from wanting to handle the exception in the HDFSWriter, where the exceptions
where being caught at the time.
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/741
I absolutely agree, this does not address the higher level issue or
severity. This is merely a stopgap to fix something that we are seeing in the
field and on the list. I made a JIRA to capture
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/742
sounds good to me. Thanks for the contribution! I'm jealous I didn't
think of it first.
---
GitHub user cestella opened a pull request:
https://github.com/apache/metron/pull/745
METRON-1171: Better validation for the SUBSTRING stellar function
## Contributor Comments
As per the comments on #742 we need better structural validation on the
SUBSTRING function.
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/740
I don't know if you saw my question, and it is hidden now so I'll restate:
What about functions that check for the GLOBAL_ flag, and may make
assumptions that do not hold with a non-z
Yes, that was me.
I was committing a pr, and got an error on this while building and testing.
it was introduced by an earlier pr. I sent an email to the list at the
time:
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/745
+1 LGTM - Thanks!
---
Github user asfgit closed the pull request at:
https://github.com/apache/metron/pull/742
---
Yeah, I fixed the dependencies_with_url problem in METRON-1169. Let's pull
that one in and the rat check one and cut a new RC. I'm -1
On Fri, Sep 8, 2017 at 7:38 PM, Matt Foley wrote:
> Couple pieces of info that may affect your vote:
>
> 1. This does not include today’s
Github user asfgit closed the pull request at:
https://github.com/apache/metron/pull/743
---
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/742#discussion_r137912849
--- Diff:
metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/StringFunctions.java
---
@@ -321,6 +321,46 @@ public
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/742#discussion_r137913052
--- Diff:
metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/StringFunctions.java
---
@@ -321,6 +321,46 @@ public
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/742#discussion_r137912941
--- Diff:
metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/StringFunctions.java
---
@@ -321,6 +321,46 @@ public
I was looking through some of the recent commits and I noticed this[1],
anybody know what the back story is there?
1:
https://github.com/apache/metron/commit/c8e84fa3be89901013168d15df38b8a58265148a
Jon
--
Jon
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/739
+1 by inspection, this looks very interesting
---
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/740
Can't we turn everything passed in into json and use the json patch stuff
to "build out' the configuration? Why not have the globals as a
map all the time?
---
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/740
Even if they don't now, I believe that is the intent of the feature
---
My guess is that what happened was otto did a pull of METRON-1061 into his
master, but it wasn't *quite* in sync with master in apache and there was a
conflict in the dependencies_with_url.csv, so that commit was made locally
to fix the conflict. both commits, the squashed and the merge conflict
I don't have an issue with this; it's good to have the explanation. I also
found that we weren't running the dependency analyzer as part of travis
(or, we thought we were, but we weren't) until METRON-1169. Thanks otto
for the explanation and jon for catching it.
On Fri, Sep 8, 2017 at 10:09
Just a FYI, I think all the relevant PRs to fix the rat check and
dependency analyzer are committed now and we can cut another RC whenever
we're ready.
On Fri, Sep 8, 2017 at 8:24 PM, Casey Stella wrote:
> Yeah, I fixed the dependencies_with_url problem in METRON-1169.
without re-writing my other reply:
I use the prepare-commit into a clean repo every time, so I believe that
this issue
*was* in the apache repo.
On September 8, 2017 at 22:08:02, Casey Stella (ceste...@gmail.com) wrote:
My guess is that what happened was otto did a pull of METRON-1061 into his
That's very weird.
On Fri, Sep 8, 2017 at 10:12 PM, Otto Fowler
wrote:
> without re-writing my other reply:
> I use the prepare-commit into a clean repo every time, so I believe that
> this issue
> *was* in the apache repo.
>
>
> On September 8, 2017 at 22:08:02, Casey
Is what I did against policy? What should I have done?
I think the committer’s guide is archived btw.
On September 8, 2017 at 22:12:27, Casey Stella (ceste...@gmail.com) wrote:
I don't have an issue with this; it's good to have the explanation. I also
found that we weren't running the
GitHub user mattf-horton opened a pull request:
https://github.com/apache/metron/pull/743
METRON-1163 RAT failures for metron-interface/metron-alerts
## Contributor Comments
Please see https://issues.apache.org/jira/browse/METRON-1163 for the
problem.
I'm not entirely
Couple pieces of info that may affect your vote:
1. This does not include today’s commits of
METRON-1162 Apply Live Messages to the Profile Debugger (nickwallen)
METRON-1077 Use column meta data end point in alerts ui (iraghumitra)
2. This does not pass RAT check
METRON-1163 turns out to be invalid, due to a bad maven settings file. It is
no longer an issue.
Just waiting for the last couple items.
On 9/7/17, 9:26 AM, "Matt Foley" wrote:
Okay. Please ping when committed.
Also, any input on
Github user justinleet commented on the issue:
https://github.com/apache/metron/pull/741
Adjusted to only catch the "Stream Closed" exception, not the general
IOException. If it's stream closed, attempt the rotate and write, otherwise
rethrow
---
Github user ottobackwards closed the pull request at:
https://github.com/apache/metron/pull/739
---
Thank you, Casey. RC3 is WITHDRAWN. I’ll post RC4 shortly, and start a new
VOTE thread.
--Matt
On 9/8/17, 7:14 PM, "Casey Stella" wrote:
Just a FYI, I think all the relevant PRs to fix the rat check and
dependency analyzer are committed now and we can cut another
Colleagues,
This is a call to vote on releasing Apache Metron 0.4.1.
The release candidate is available at
https://dist.apache.org/repos/dist/dev/metron/0.4.1-RC4/
Full list of changes in this release:
https://dist.apache.org/repos/dist/dev/metron/0.4.1-RC4/CHANGES
The github tag to be voted
So, generally the goal is to commit the minimal set of commits squashed by
contributor. For situations that are wacky and end up with a conflict, I
don't think it's a problem. No worries, IMO.
On Fri, Sep 8, 2017 at 10:29 PM, Otto Fowler
wrote:
> Is what I did against
94 matches
Mail list logo