Disable Metron parser output writer entirely

[Ali Nazemian]

Hi All,

I am trying to investigate whether we can disable a Metron parser output
writer entirely and manage it via KAFKA_PUT Stellar function instead.
First, is it possible via configuration? Second, will be any performance
difference between normal Kafka writer and the Stellar version of it
(KAFKA_PUT).

Regards,
Ali

[GitHub] metron pull request #922: METRON-1441: Create complementary Solr schemas for...

[ottobackwards]

Github user ottobackwards commented on a diff in the pull request:

https://github.com/apache/metron/pull/922#discussion_r165548937
  
--- Diff: 
metron-platform/metron-solr/src/test/java/org/apache/metron/solr/schema/SchemaTranslatorTest.java
 ---
@@ -0,0 +1,188 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.solr.schema;
+
+import com.google.common.base.Splitter;
+import com.google.common.collect.Iterables;
+import org.adrianwalker.multilinestring.Multiline;
+import org.apache.metron.common.configuration.writer.WriterConfiguration;
+import org.apache.metron.common.utils.JSONUtils;
+import org.apache.metron.integration.UnableToStartException;
+import org.apache.metron.solr.integration.components.SolrComponent;
+import org.apache.metron.solr.writer.SolrWriter;
+import org.json.simple.JSONObject;
+import org.junit.Assert;
+import org.junit.Test;
+
+import java.io.IOException;
+import java.io.PrintWriter;
+import java.io.StringWriter;
+import java.util.*;
+
+public class SchemaTranslatorTest {
+
+  /**

+{"adapter.threatinteladapter.end.ts":"1517499201357","bro_timestamp":"1517499194.7338","ip_dst_port":8080,"enrichmentsplitterbolt.splitter.end.ts":"1517499201202","enrichmentsplitterbolt.splitter.begin.ts":"1517499201200","adapter.hostfromjsonlistadapter.end.ts":"1517499201207","adapter.geoadapter.begin.ts":"1517499201209","uid":"CUrRne3iLIxXavQtci","trans_depth":143,"protocol":"http","original_string":"HTTP
 | id.orig_p:50451 method:GET request_body_len:0 id.resp_p:8080 
uri:\/api\/v1\/clusters\/metron_cluster\/services\/KAFKA\/components\/KAFKA_BROKER?fields=metrics\/kafka\/server\/BrokerTopicMetrics\/AllTopicsBytesInPerSec\/1MinuteRate[1484165330,1484168930,15],metrics\/kafka\/server\/BrokerTopicMetrics\/AllTopicsBytesOutPerSec\/1MinuteRate[1484165330,1484168930,15],metrics\/kafka\/server\/BrokerTopicMetrics\/AllTopicsMessagesInPerSec\/1MinuteRate[1484165330,1484168930,15],metrics\/kafka\/controller\/KafkaController\/ActiveControllerCount[1484165330,1484168930,15],metrics\/kafk
 
a\/controller\/ControllerStats\/LeaderElectionRateAndTimeMs\/1MinuteRate[1484165330,1484168930,15],metrics\/kafka\/controller\/ControllerStats\/UncleanLeaderElectionsPerSec\/1MinuteRate[1484165330,1484168930,15],metrics\/kafka\/server\/ReplicaFetcherManager\/Replica-MaxLag[1484165330,1484168930,15],metrics\/kafka\/server\/ReplicaManager\/PartitionCount[1484165330,1484168930,15],metrics\/kafka\/server\/ReplicaManager\/UnderReplicatedPartitions[1484165330,1484168930,15],metrics\/kafka\/server\/ReplicaManager\/LeaderCount[1484165330,1484168930,15]=null_padding&_=1484168930776
 tags:[] uid:CUrRne3iLIxXavQtci referrer:http:\/\/node1:8080\/ trans_depth:143 
host:node1 id.orig_h:192.168.66.1 response_body_len:0 user_agent:Mozilla\/5.0 
(Macintosh; Intel Mac OS X 10_12_2) AppleWebKit\/537.36 (KHTML, like Gecko) 
Chrome\/55.0.2883.95 Safari\/537.36 ts:1517499194.7338 
id.resp_h:192.168.66.121","ip_dst_addr":"192.168.66.121","threatinteljoinbolt.joiner.ts":"1517499201359","host":"node1","en
 
richmentjoinbolt.joiner.ts":"1517499201212","adapter.hostfromjsonlistadapter.begin.ts":"1517499201206","threatintelsplitterbolt.splitter.begin.ts":"1517499201215","ip_src_addr":"192.168.66.1","user_agent":"Mozilla\/5.0
 (Macintosh; Intel Mac OS X 10_12_2) AppleWebKit\/537.36 (KHTML, like Gecko) 
Chrome\/55.0.2883.95 
Safari\/537.36","timestamp":1517499194733,"method":"GET","request_body_len":0,"uri":"\/api\/v1\/clusters\/metron_cluster\/services\/KAFKA\/components\/KAFKA_BROKER?fields=metrics\/kafka\/server\/BrokerTopicMetrics\/AllTopicsBytesInPerSec\/1MinuteRate[1484165330,1484168930,15],metrics\/kafka\/server\/BrokerTopicMetrics\/AllTopicsBytesOutPerSec\/1MinuteRate[1484165330,1484168930,15],metrics\/kafka\/server\/BrokerTopicMetrics\/AllTopicsMessagesInPerSec\/1MinuteRate[1484165330,1484168930,15],metrics\/kafka\/controller\/KafkaController\/ActiveControllerCount[1484165330,1484168930,15],metrics\/kafka\/controller\/ControllerStats\/LeaderElectionRateAndTimeMs\/1MinuteRate[148416533
 

[GitHub] metron pull request #922: METRON-1441: Create complementary Solr schemas for...

[ottobackwards]

Github user ottobackwards commented on a diff in the pull request:

https://github.com/apache/metron/pull/922#discussion_r165548986
  
--- Diff: 
metron-platform/metron-solr/src/test/java/org/apache/metron/solr/schema/SchemaTranslatorTest.java
 ---
@@ -0,0 +1,188 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.solr.schema;
+
+import com.google.common.base.Splitter;
+import com.google.common.collect.Iterables;
+import org.adrianwalker.multilinestring.Multiline;
+import org.apache.metron.common.configuration.writer.WriterConfiguration;
+import org.apache.metron.common.utils.JSONUtils;
+import org.apache.metron.integration.UnableToStartException;
+import org.apache.metron.solr.integration.components.SolrComponent;
+import org.apache.metron.solr.writer.SolrWriter;
+import org.json.simple.JSONObject;
+import org.junit.Assert;
+import org.junit.Test;
+
+import java.io.IOException;
+import java.io.PrintWriter;
+import java.io.StringWriter;
+import java.util.*;
+
--- End diff --

Shouldn't this be an integration test?


---

Re: [DISCUSS] Persistence store for user profile settings

[Michael Miklavcic]

Personally, I'd be in favor of something like Maria DB as an open source
repo. Or any other ansi sql store. On the positive side, it should mesh
seamlessly with ORM tools. And the schema for this should be pretty
vanilla, I'd imagine. I might even consider skipping ORM for straight JDBC
and simple command scripts in Java for something this small. I'm not
worried so much about migrations of this sort. Large scale DBs can get
involved with major schema changes, but thats usually when the datastore is
a massive set of tables with complex relationships, at least in my
experience.

We could also use hbase, which probably wouldn't be that hard either, but
there may be more boilerplate to write for the client as compared to
standard SQL. But I'm assuming we could reuse a fair amount of existing
code from our enrichments. One additional reason in favor of hbase might be
data replication. For a SQL instance we'd probably recommend a RAID store
or backup procedure, but we get that pretty easy with hbase too.

On Feb 1, 2018 2:45 PM, "Casey Stella"  wrote:

> So, I'll answer your question with some questions:
>
>- No matter the data store we use upgrading will take some care, right?
>- Do we currently depend on a RDBMS anywhere?  I want to say that we do
>in the REST layer already, right?
>- If we don't use a RDBMs, what's the other option?  What are the pros
>and cons?
>- Have we considered non-server offline persistent solutions (e.g.
>https://www.html5rocks.com/en/features/storage)?
>
>
>
> On Thu, Feb 1, 2018 at 9:11 AM, Ryan Merriman  wrote:
>
> > There is currently a PR up for review that allows a user to configure and
> > save the list of facet fields that appear in the left column of the
> Alerts
> > UI:  https://github.com/apache/metron/pull/853.  The REST layer has ORM
> > support which means we can store those in a relational database.
> >
> > However I'm not 100% sure this is the best place to keep this.  As we add
> > more use cases like this the backing tables in the RDBMS will need to be
> > managed.  This could make upgrading more tedious and error-prone.  Is
> there
> > are a better way to store this, assuming we can leverage a component
> that's
> > already included in our stack?
> >
> > Ryan
> >
>

[GitHub] metron pull request #911: METRON-1419: Create a SolrDao

[merrimanr]

Github user merrimanr closed the pull request at:

https://github.com/apache/metron/pull/911


---

Re: [DISCUSS] Persistence store for user profile settings

[Casey Stella]

So, I'll answer your question with some questions:

   - No matter the data store we use upgrading will take some care, right?
   - Do we currently depend on a RDBMS anywhere?  I want to say that we do
   in the REST layer already, right?
   - If we don't use a RDBMs, what's the other option?  What are the pros
   and cons?
   - Have we considered non-server offline persistent solutions (e.g.
   https://www.html5rocks.com/en/features/storage)?



On Thu, Feb 1, 2018 at 9:11 AM, Ryan Merriman  wrote:

> There is currently a PR up for review that allows a user to configure and
> save the list of facet fields that appear in the left column of the Alerts
> UI:  https://github.com/apache/metron/pull/853.  The REST layer has ORM
> support which means we can store those in a relational database.
>
> However I'm not 100% sure this is the best place to keep this.  As we add
> more use cases like this the backing tables in the RDBMS will need to be
> managed.  This could make upgrading more tedious and error-prone.  Is there
> are a better way to store this, assuming we can leverage a component that's
> already included in our stack?
>
> Ryan
>

[GitHub] metron pull request #921: METRON-1441: Create complementary Solr schemas for...

[cestella]

Github user cestella closed the pull request at:

https://github.com/apache/metron/pull/921


---

[GitHub] metron pull request #922: METRON-1441: Create complementary Solr schemas for...

[cestella]

GitHub user cestella opened a pull request:

https://github.com/apache/metron/pull/922

METRON-1441: Create complementary Solr schemas for the main sensors

## Contributor Comments
We have ES templates for bro, snort, yaf, and error, we need corresponding 
solr schemas for these collections.

Right now this is tested via the SchemaTranslatorTest, which spins up the 
current version of Solr loads the schemas and writes sample data and ensures 
they come back properly. 

Note: This is an intermediate move to getting full solr support.  These 
schemas will be refined more, likely.

## Pull Request Checklist

Thank you for submitting a contribution to Apache Metron.  
Please refer to our [Development 
Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235)
 for the complete guide to follow for contributions.  
Please refer also to our [Build Verification 
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview)
 for complete smoke testing guides.  


In order to streamline the review of the contribution we ask you follow 
these guidelines and ask you to double check the following:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to 
be created at [Metron 
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

 Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.
It is also recommended that [travis-ci](https://travis-ci.org) is set up 
for your personal repository such that your branches are built there before 
submitting a pull request.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/cestella/incubator-metron SOLR_METRON-1441

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/metron/pull/922.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #922


commit 6bb30af9d2005414e3ee44c0bdb0ea14540ce13c
Author: cstella 
Date:   2018-02-01T21:33:56Z

METRON-1441: Create complementary Solr schemas for the main sensors




---

[GitHub] metron issue #911: METRON-1419: Create a SolrDao

[cestella]

Github user cestella commented on the issue:

https://github.com/apache/metron/pull/911
  
actually, it may be that feature branch PRs don't automatically close..or 
hmm, not sure.  Anyway, close it please :)


---

[GitHub] metron issue #911: METRON-1419: Create a SolrDao

[cestella]

Github user cestella commented on the issue:

https://github.com/apache/metron/pull/911
  
well, I botched the stupid commit message and put `apache/metron` when I 
meant `apache/feature/METRON-1416-upgrade-solr`.  Can you close this PR 
@merrimanr ?


---

[GitHub] metron issue #921: METRON-1441: Create complementary Solr schemas for the ma...

[cestella]

Github user cestella commented on the issue:

https://github.com/apache/metron/pull/921
  
I know that commit history looks pretty awful, but it's because I branched 
off of @merrimanr 's SolrDao PR initially.  Now that PR is in the branch, so 
the comparison should be more sensible.


---

[GitHub] metron issue #918: METRON-1436: Manually Install Solr Cloud in Full Dev

[justinleet]

Github user justinleet commented on the issue:

https://github.com/apache/metron/pull/918
  
I got full dev spinning spun up with this, and it looks good.  I think the 
latest couple comments are still to be addressed, but otherwise good to go.


---

[GitHub] metron pull request #921: METRON-1441: Create complementary Solr schemas for...

[cestella]

GitHub user cestella opened a pull request:

https://github.com/apache/metron/pull/921

METRON-1441: Create complementary Solr schemas for the main sensors

## Contributor Comments
We have ES templates for bro, snort, yaf, and error, we need corresponding 
solr schemas for these collections.

Right now this is tested via the SchemaTranslatorTest, which spins up the 
current version of Solr loads the schemas and writes sample data and ensures 
they come back properly. 

Note: This is an intermediate move to getting full solr support.  These 
schemas will be refined more, likely.

## Pull Request Checklist

Thank you for submitting a contribution to Apache Metron.  
Please refer to our [Development 
Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235)
 for the complete guide to follow for contributions.  
Please refer also to our [Build Verification 
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview)
 for complete smoke testing guides.  


In order to streamline the review of the contribution we ask you follow 
these guidelines and ask you to double check the following:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to 
be created at [Metron 
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

 Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.
It is also recommended that [travis-ci](https://travis-ci.org) is set up 
for your personal repository such that your branches are built there before 
submitting a pull request.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/cestella/incubator-metron solr_schemas

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/metron/pull/921.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #921


commit c86421739f230cfc23b2a5092af53a32348dbaef
Author: merrimanr 
Date:   2018-01-24T16:35:33Z

initial commit

commit 76455986a14d12ed720a83c091616a1904ffb2b1
Author: merrimanr 
Date:   2018-01-24T19:07:46Z

initial commit

commit f996ee1a3069116cf1063e70c7d4a2b562da9b97
Author: merrimanr 
Date:   2018-01-24T19:16:15Z

Merge branch 'METRON-1429' into solr-dao

# Conflicts:
#   
metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/integration/ElasticsearchSearchIntegrationTest.java
#   
metron-platform/metron-indexing/src/test/java/org/apache/metron/indexing/dao/SearchIntegrationTest.java

commit a29ed14c39a79ce5918212baa9b075706319e4de
Author: merrimanr 
Date:   2018-01-24T20:31:11Z

added back metaalert test data

commit 92cc5640e7cacf4cc3b4941c3bc7d559aa7b8380
Author: merrimanr 
Date:   2018-01-24T20:31:49Z

Merge branch 'METRON-1429' into solr-dao

commit 3a17b4d9d18efe4431197521124b13cb0dc4f9e3
Author: merrimanr 
Date:   2018-01-24T22:32:52Z

resolved merge conflicts

commit 5c6293dd5302cddf3f212a982fed8c2d0ec2e391
Author: merrimanr 
Date:   2018-01-24T22:33:53Z

Merge remote-tracking branch 'mirror/master' into METRON-1429

commit bb8efc16f60147ed8198604923c391c360376859
Author: merrimanr 
Date:   2018-01-24T22:49:36Z

Merge branch 'METRON-1429' into solr-dao

commit 016403323cda9959c8edcdbe0e6a6e8d7cda9340
Author: merrimanr 
Date:   2018-01-25T16:37:10Z

Merge remote-tracking branch 'mirror/master' into solr-dao

# Conflicts:
#   
metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/integration/ElasticsearchSearchIntegrationTest.java
#   
metron-platform/metron-indexing/src/test/java/org/apache/metron/indexing/dao/SearchIntegrationTest.java

commit d4c99e1521f7c638ab81d388b0cb8c55cc548561
Author: merrimanr 
Date:   2018-01-26T22:21:38Z

dao refactor and group implementation

commit 8fc77dff00a695f09ea6777aada9e23f7e89f6ee
Author: merrimanr 
Date:   2018-01-26T22:25:15Z

Merge remote-tracking branch 'mirror/master' into solr-dao

# Conflicts:
#   
metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/integration/ElasticsearchSearchIntegrationTest.java
#   

[GitHub] metron issue #911: METRON-1419: Create a SolrDao

[justinleet]

Github user justinleet commented on the issue:

https://github.com/apache/metron/pull/911
  
I looked over the changes and refactor.  Thanks a lot for taking care of 
that, it looks a lot better and is definitely easier to grok, imo, than the old 
way.

I'm +1 on merging this into the feature branch.


---

[GitHub] metron pull request #919: METRON-1439: Turn off git pager in platform-info s...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/metron/pull/919


---

[GitHub] metron pull request #920: METRON-1438 Move SHELL functions from metron-manag...

[ottobackwards]

GitHub user ottobackwards opened a pull request:

https://github.com/apache/metron/pull/920

METRON-1438 Move SHELL functions from metron-management to stellar-common

Part of making stellar more modular and stand alone is consolidation of 
stellar functionality scattered through metron into stellar common.   That 
should be done if the functionality is generally useful and not metron 
specific. 

Where the functionality *is* metron specific, we should look to factor such 
functionality as to extract generally useful functionality.

In the case of the SHELL- namespace, it is clearly not metron specific.

This pr moves this functionality from the metron-management module to 
stellar common.

## Testing
- build and tests should run as normal
- shell functions such as those in the metron-management readme examples 
should run
- functions should work as before


### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to 
be created at [Metron 
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?


### For code changes:
- [x] Have you included steps to reproduce the behavior or problem that is 
being changed or addressed?
- [x] Have you included steps or a guide to how the change may be verified 
and tested manually?
- [x] Have you ensured that the full suite of tests and checks have been 
executed in the root metron folder via:
  ```
  mvn -q clean integration-test install && 
dev-utilities/build-utils/verify_licenses.sh 
  ```

- [na] Have you written or updated unit tests and or integration tests to 
verify your changes?
- [na] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [na] Have you verified the basic functionality of the build by building 
and running locally with Vagrant full-dev environment or the equivalent?

### For documentation related changes:
- [x] Have you ensured that format looks appropriate for the output in 
which it is rendered by building and verifying the site-book? If not then run 
the following commands and the verify changes via 
`site-book/target/site/index.html`:



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ottobackwards/metron stellar-move-shell-funcs

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/metron/pull/920.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #920


commit fe93fc40d94cc6148667eccc36853a2dc9925546
Author: Otto Fowler 
Date:   2018-02-01T14:28:46Z

move ShellFunctions to stellar-common

commit 38775033240c7c174c99b10ffb5087c39b23672a
Author: Otto Fowler 
Date:   2018-02-01T15:33:42Z

cleanup and refactor for failed tests where PausableInput did not unpause

commit 7af3d347567fe967c9c2ea1191c121a470d2bf34
Author: Otto Fowler 
Date:   2018-02-01T18:28:43Z

fix readme




---

[GitHub] metron issue #919: METRON-1439: Turn off git pager in platform-info script

[ottobackwards]

Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/919
  
No, I am sorry, I thought that was understood.  I have not reviewed, but I 
would go ahead based on @cestella 


---

[GitHub] metron issue #919: METRON-1439: Turn off git pager in platform-info script

[justinleet]

Github user justinleet commented on the issue:

https://github.com/apache/metron/pull/919
  
@ottobackwards Any objections to having the discussion outside of this and 
merging this in as-is?


---

[GitHub] metron issue #919: METRON-1439: Turn off git pager in platform-info script

[cestella]

Github user cestella commented on the issue:

https://github.com/apache/metron/pull/919
  
so, I'm +1 on this and we probably want a discussion outside of this about 
where to move `platform-info.sh`


---

[GitHub] metron issue #919: METRON-1439: Turn off git pager in platform-info script

[ottobackwards]

Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/919
  
this is a user utility though, not a dev one.  I would create a new 
support-scripts folder


---

[GitHub] metron issue #919: METRON-1439: Turn off git pager in platform-info script

[nickwallen]

Github user nickwallen commented on the issue:

https://github.com/apache/metron/pull/919
  
> Sidenote, do we want to move this script to the dev-utilities dir?

I do think it would be good to move that script.  Not sure where under 
dev-utilities though.  And we can tackle as separate PR, if you like.




---

[GitHub] metron issue #919: METRON-1439: Turn off git pager in platform-info script

[nickwallen]

Github user nickwallen commented on the issue:

https://github.com/apache/metron/pull/919
  
+1 works great for me.  Thanks


---

[GitHub] metron issue #919: METRON-1439: Turn off git pager in platform-info script

[justinleet]

Github user justinleet commented on the issue:

https://github.com/apache/metron/pull/919
  
Sidenote, do we want to move this script to the dev-utilities dir?


---

[GitHub] metron pull request #919: METRON-1439: Turn off git pager in platform-info s...

[justinleet]

GitHub user justinleet opened a pull request:

https://github.com/apache/metron/pull/919

METRON-1439: Turn off git pager in platform-info script

## Contributor Comments
Just added a couple --no-pager args.  Before you'd have to progress past 
the couple git pages, now they should just be part of the output.

Output should look something like this when run:
```
{10:08}~/Documents/workspace/metron:METRON-1439 ✗ ➭  
metron-deployment/scripts/platform-info.sh
Metron 0.4.3
--
* METRON-1439
--
commit a285b83ee0153cecf24132b4aed70250040a83b0 (HEAD -> METRON-1439, 
upstream/master, origin/master, origin/HEAD, 
apache/feature/METRON-1416-upgrade-solr, master, 
feature/METRON-1416-upgrade-solr)
Author: nickwallen 
Date:   Tue Jan 30 09:31:53 2018 -0500

METRON-1432 JDK Install Fails on Ubuntu Development Environment  
(nickwallen) closes apache/metron#913
--
 metron-deployment/scripts/platform-info.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
--
ansible 2.2.2.0
  config file =
  configured module search path = Default w/o overrides
--
Vagrant 1.8.1
--
Python 2.7.10
--
```

## Pull Request Checklist

Thank you for submitting a contribution to Apache Metron.  
Please refer to our [Development 
Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235)
 for the complete guide to follow for contributions.  
Please refer also to our [Build Verification 
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview)
 for complete smoke testing guides.  


In order to streamline the review of the contribution we ask you follow 
these guidelines and ask you to double check the following:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to 
be created at [Metron 
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?
- [x] Have you included steps or a guide to how the change may be verified 
and tested manually?

 Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.
It is also recommended that [travis-ci](https://travis-ci.org) is set up 
for your personal repository such that your branches are built there before 
submitting a pull request.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/justinleet/metron METRON-1439

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/metron/pull/919.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #919


commit 0448218d4ef5e1321fac42350e89edd5d8131dc8
Author: justinjleet 
Date:   2018-02-01T15:09:10Z

Adding --no-pager args




---

[GitHub] metron issue #917: METRON-1435: Management UI cannot save json objects in ad...

[cestella]

Github user cestella commented on the issue:

https://github.com/apache/metron/pull/917
  
+1 by inspection, great work!


---

[GitHub] metron issue #579: METRON-941 fix PaloAltoParser

[ottobackwards]

Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/579
  
@ctramnitz thank you!  Let us know where you are at and if we can help


---

[GitHub] metron issue #579: METRON-941 fix PaloAltoParser

[ctramnitz]

Github user ctramnitz commented on the issue:

https://github.com/apache/metron/pull/579
  
Rebasing to master to see where we are. If this comes back clean please 
don't merge yet, I want to add 8.0 log format first.


---

[DISCUSS] Persistence store for user profile settings

[Ryan Merriman]

There is currently a PR up for review that allows a user to configure and
save the list of facet fields that appear in the left column of the Alerts
UI:  https://github.com/apache/metron/pull/853.  The REST layer has ORM
support which means we can store those in a relational database.

However I'm not 100% sure this is the best place to keep this.  As we add
more use cases like this the backing tables in the RDBMS will need to be
managed.  This could make upgrading more tedious and error-prone.  Is there
are a better way to store this, assuming we can leverage a component that's
already included in our stack?

Ryan

[GitHub] metron issue #865: METRON-1212 The bundle System and Maven Plugin (Feature B...

[JonZeolla]

Github user JonZeolla commented on the issue:

https://github.com/apache/metron/pull/865
  
Ping


---

[GitHub] metron issue #915: METRON-1433: Only emit debugging timing fields in enrichm...

[mraliagha]

Github user mraliagha commented on the issue:

https://github.com/apache/metron/pull/915
  
@cestella Definitely there will be value for having an ability to turn it 
on and off especially since you have already implemented that. However, won't 
be a sort of premature disk optimization? We are storing original_string and 
lots of other things that generally we may not really use them. Are those 
timestamp fields really troublemaker?


---