[jira] [Created] (METRON-1201) Support for repositories of Metron Extensions
Otto Fowler created METRON-1201: --- Summary: Support for repositories of Metron Extensions Key: METRON-1201 URL: https://issues.apache.org/jira/browse/METRON-1201 Project: Metron Issue Type: New Feature Reporter: Otto Fowler Metron should have the ability to have extensions available to the system from either local or remote repositories. At that point, we will support extension installation by: * file * local repo * [n] remote repos The system should support signing the extensions, and registration of for extension developers of some type. Submittal and acceptance ( curation ) should be considered if the repo is hosted by metron. This idea needs some flushing out, and should be broken out further -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (METRON-1200) Configuration only parser extensions should not have empty src/main directories
[ https://issues.apache.org/jira/browse/METRON-1200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Otto Fowler updated METRON-1200: Summary: Configuration only parser extensions should not have empty src/main directories (was: [FEATURE BRANCH] Configuration only parser extensions should not have empty src/main directories) > Configuration only parser extensions should not have empty src/main > directories > > > Key: METRON-1200 > URL: https://issues.apache.org/jira/browse/METRON-1200 > Project: Metron > Issue Type: New Feature >Reporter: Otto Fowler > Labels: metron-feature-canidate, > metron-feature-extensions-parsers > > Verify that squid and yaf are correct. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (METRON-1200) [FEATURE BRANCH] Configuration only parser extensions should not have empty src/main directories
Otto Fowler created METRON-1200: --- Summary: [FEATURE BRANCH] Configuration only parser extensions should not have empty src/main directories Key: METRON-1200 URL: https://issues.apache.org/jira/browse/METRON-1200 Project: Metron Issue Type: New Feature Reporter: Otto Fowler Verify that squid and yaf are correct. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (METRON-1189) Add alert escalation to the Alerts UI
[ https://issues.apache.org/jira/browse/METRON-1189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175337#comment-16175337 ] ASF GitHub Bot commented on METRON-1189: Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/762 I researched the saveRefreshState() and restoreRefreshState() addition and it uncovered a bug which then uncovered a problem with how our UI works. The bug was that the selectedAlerts field (checkboxes in the list) was not getting cleared on any of the bulk actions. As refreshes happened the checkboxes would disappear and be replaced by new results but that that field would constantly grow because they were never unchecked on the previous page. It was benign before this PR but is now causing a lot of duplicate REST requests. We need to figure out the right way to clear these when a bulk action finishes but there is also another issue. As I was testing for this I realized when I set the refresh rate to a high rate I had to click quickly or the page would change and I would lose my selections. What @iraghumitra mentions above is a pause while the REST requests are being processed but I think it makes sense to pause even earlier, whenever you have a checkbox checked. I can obviously get it to work if I pause it but I think that's a better UX. > Add alert escalation to the Alerts UI > - > > Key: METRON-1189 > URL: https://issues.apache.org/jira/browse/METRON-1189 > Project: Metron > Issue Type: New Feature >Reporter: Ryan Merriman >Assignee: Ryan Merriman > > The PR is a follow on to https://issues.apache.org/jira/browse/METRON-1127. > We need a way to escalate alerts in the alerts UI so that a user can selects > alerts and send them to the escalate REST endpoint. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (METRON-1199) Create new version of the Squid Tutorial Fundamentals page with Parser Extensions
Otto Fowler created METRON-1199: --- Summary: Create new version of the Squid Tutorial Fundamentals page with Parser Extensions Key: METRON-1199 URL: https://issues.apache.org/jira/browse/METRON-1199 Project: Metron Issue Type: New Feature Reporter: Otto Fowler Metron Tutorial - Fundamentals Part 1: Creating a New Telemetry We should have a version of this using parser extensions. This version should cover creating a parser from archetype, and installing with the ui. Then have the same as the other tutorial basically -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (METRON-1182) Refactor Code in alert list to accommodate new view types
[ https://issues.apache.org/jira/browse/METRON-1182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175190#comment-16175190 ] ASF GitHub Bot commented on METRON-1182: Github user asfgit closed the pull request at: https://github.com/apache/metron/pull/756 > Refactor Code in alert list to accommodate new view types > - > > Key: METRON-1182 > URL: https://issues.apache.org/jira/browse/METRON-1182 > Project: Metron > Issue Type: Improvement >Reporter: RaghuMitra >Assignee: RaghuMitra > > Decouple the filters, save search, search and other options from the table > that displays data. > This would help us to add a different view type for displaying tabular data. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (METRON-1182) Refactor Code in alert list to accommodate new view types
[ https://issues.apache.org/jira/browse/METRON-1182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175180#comment-16175180 ] ASF GitHub Bot commented on METRON-1182: Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/756 Lots of improvements in this PR, I like it. Spun this up in full dev and everything continues to work. Nice job. +1 > Refactor Code in alert list to accommodate new view types > - > > Key: METRON-1182 > URL: https://issues.apache.org/jira/browse/METRON-1182 > Project: Metron > Issue Type: Improvement >Reporter: RaghuMitra >Assignee: RaghuMitra > > Decouple the filters, save search, search and other options from the table > that displays data. > This would help us to add a different view type for displaying tabular data. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (METRON-938) "service metron-rest start " does not work on CentOS 7.
[ https://issues.apache.org/jira/browse/METRON-938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174952#comment-16174952 ] ASF GitHub Bot commented on METRON-938: --- Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/757 Spun it up again in full dev, setup MySQL and everything works as it should. Thanks so much @justinleet for fixing this! Turned out great. +1 > "service metron-rest start " does not work on CentOS 7. > - > > Key: METRON-938 > URL: https://issues.apache.org/jira/browse/METRON-938 > Project: Metron > Issue Type: Bug >Affects Versions: 0.4.0 > Environment: Metron 0.4.0 on CentOS 7. >Reporter: Laurens Vets >Priority: Minor > > I suppose due to the change to systemd, the command "service metron-rest > start " does not work anymore in CentOS 7. > Changing "METRON_JDBC_PASSWORD="$2"" to "METRON_JDBC_PASSWORD=" PASSWORD>"" in "/etc/rc.d/init.d/metron-rest" seems to fix this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (METRON-1161) Add ability to edit parser command line options in the management UI
[ https://issues.apache.org/jira/browse/METRON-1161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174843#comment-16174843 ] ASF GitHub Bot commented on METRON-1161: Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/737 Changing numWorkers and numAckers default values from null to 1 had the unintended consequences of breaking the ParserTopologyCliTest. This is because they can no longer be overriden by setting the topology.workers and topology.acker.executors properties in stormConfig. I believe this is the correct behaviour because numWorkers and numAckers are top-level properties and should take precedence. If this is not correct then I would argue that we shouldn't even have them. Can you verify this @cestella? > Add ability to edit parser command line options in the management UI > > > Key: METRON-1161 > URL: https://issues.apache.org/jira/browse/METRON-1161 > Project: Metron > Issue Type: New Feature >Reporter: Ryan Merriman >Assignee: Ryan Merriman > > This is a follow on to https://issues.apache.org/jira/browse/METRON-1134 that > exposes the new zookeeper parser command line options in the Management UI. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (METRON-1198) Pycapa - No such configuration property: "sasl.kerberos.principal"
[
https://issues.apache.org/jira/browse/METRON-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174764#comment-16174764
]
ASF GitHub Bot commented on METRON-1198:
GitHub user nickwallen opened a pull request:
https://github.com/apache/metron/pull/769
METRON-1198: Pycapa - No such configuration property:
"sasl.kerberos.principal"
When running Pycapa in a Kerberized environment, but without a version of
Librdkafka built with SASL support, it can produce error messages that
look-like the following.
```
KafkaError{code=_INVALID_ARG,val=-186,str="No such configuration property:
"sasl.kerberos.principal""}
```
This can happen when a user accidentally installs multiple version of
Librdkafka and the version that the Python interpreter links to is the one
without SASL support. I am going to update the README to doc this specific
condition.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/nickwallen/metron patch-2
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metron/pull/769.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #769
commit beaeca920ebb2de7131fdb2e0e036f016874d530
Author: Nick Allen
Date: 2017-09-21T13:39:18Z
METRON-1198: Pycapa - No such configuration property:
"sasl.kerberos.principal"
> Pycapa - No such configuration property: "sasl.kerberos.principal"
> --
>
> Key: METRON-1198
> URL: https://issues.apache.org/jira/browse/METRON-1198
> Project: Metron
> Issue Type: Improvement
>Affects Versions: 0.4.1
>Reporter: Nick Allen
>Assignee: Nick Allen
> Fix For: Next + 1
>
>
> When running pycapa in a Kerberized environment, but without a version of
> librdkafka built with SASL support, it can produce error messages that
> look-like the following.
> {code}
> KafkaError{code=_INVALID_ARG,val=-186,str="No such configuration property:
> "sasl.kerberos.principal""}
> {code}
> This can happen when a user accidentally installs multiple version of
> librdkafka and the version that the Python interpreter links to is the one
> without SASL support.
> I am going to update the README to doc this specific condition.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (METRON-1198) Pycapa - No such configuration property: "sasl.kerberos.principal"
Nick Allen created METRON-1198:
--
Summary: Pycapa - No such configuration property:
"sasl.kerberos.principal"
Key: METRON-1198
URL: https://issues.apache.org/jira/browse/METRON-1198
Project: Metron
Issue Type: Improvement
Affects Versions: 0.4.1
Reporter: Nick Allen
Assignee: Nick Allen
Fix For: Next + 1
When running pycapa in a Kerberized environment, but without a version of
librdkafka built with SASL support, it can produce error messages that
look-like the following.
{code}
KafkaError{code=_INVALID_ARG,val=-186,str="No such configuration property:
"sasl.kerberos.principal""}
{code}
This can happen when a user accidentally installs multiple version of
librdkafka and the version that the Python interpreter links to is the one
without SASL support.
I am going to update the README to doc this specific condition.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (METRON-1188) Ambari global configuration management broken
[ https://issues.apache.org/jira/browse/METRON-1188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174693#comment-16174693 ] ASF GitHub Bot commented on METRON-1188: Github user justinleet commented on the issue: https://github.com/apache/metron/pull/760 Thanks for making the adjustments, @mmiklavc. I'm +1 on this. It's a great step forward and gives a good foundation for future improvements we want to make. > Ambari global configuration management broken > - > > Key: METRON-1188 > URL: https://issues.apache.org/jira/browse/METRON-1188 > Project: Metron > Issue Type: Bug >Reporter: Michael Miklavcic >Assignee: Michael Miklavcic > > Ambari currently ignores values from the UI when managing the global.json > file. These values are hard-coded in the status_params.py file as part of the > mpack. In addition, Ambari restarts are clobbering configuration changes in > Zookeeper. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (METRON-1189) Add alert escalation to the Alerts UI
[ https://issues.apache.org/jira/browse/METRON-1189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174621#comment-16174621 ] ASF GitHub Bot commented on METRON-1189: GitHub user iraghumitra opened a pull request: https://github.com/apache/metron/pull/768 Metron 1123: Add group by option using faceted search capabilities of metron-rest-api ## Contributor Comments This PR extends the Metron Alerts GUI capabilities to do group by on Alerts Data The PR is on top of METRON-1189 The group by buttons appear on top of the table. When user selects a group, the table view changes to a tree view with the values of the groups as the root of the tree. The individual alerts would be shown as the leaf nodes of the tree. Tree view has all the following features - Search will search through all the groups - Users can click on the data inside the table to search - Users can select and table inside tree to view the details - Users can escalate the alerts from the table inside the tree - Tree view is refreshed automatically depending on the polling setting in the settings - A cumulative score and number of alerts in a group is shown next to group name - Table inside a group can be paginated - The column names for the alerts table in the tree can be configured via configure column pane The table config settings is moved next to settings on the GUI as it makes more meaning there.    ## Testing E2E test cases are available to test groups functionality. From root of the project ``` cd metron-interface/metron-alerts sh ./scripts/start-server-for-e2e.sh ``` In othere shell run the test cases ``` npm run e2e ``` All the features listed in contributor comments can be verified manually ## Pull Request Checklist Thank you for submitting a contribution to Apache Metron. Please refer to our [Development Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235) for the complete guide to follow for contributions. Please refer also to our [Build Verification Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview) for complete smoke testing guides. In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel). - [x] Does your PR title start with METRON- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? ### For code changes: - [x] Have you included steps to reproduce the behavior or problem that is being changed or addressed? - [x] Have you included steps or a guide to how the change may be verified and tested manually? - [x] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via: ``` mvn -q clean integration-test install && build_utils/verify_licenses.sh ``` - [x] Have you written or updated unit tests and or integration tests to verify your changes? - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`: ``` cd site-book mvn site ``` Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. It is also recommended that [travis-ci](https://travis-ci.org) is set up for your personal repository such that your branches are
