[jira] [Commented] (METRON-1442) Rest endpoints for operations on Indexing topology is broken after splitting the indexing topology
[ https://issues.apache.org/jira/browse/METRON-1442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16351218#comment-16351218 ] ASF GitHub Bot commented on METRON-1442: Github user MohanDV commented on the issue: https://github.com/apache/metron/pull/923 @cestella I merged your PR > Rest endpoints for operations on Indexing topology is broken after splitting > the indexing topology > -- > > Key: METRON-1442 > URL: https://issues.apache.org/jira/browse/METRON-1442 > Project: Metron > Issue Type: Bug >Reporter: Mohan >Assignee: Mohan >Priority: Major > > Rest endpoints for operations on Indexing topology is broken after splitting > the indexing topology into random_access_indexing and batch_indexing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1442) Rest endpoints for operations on Indexing topology is broken after splitting the indexing topology
[ https://issues.apache.org/jira/browse/METRON-1442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16351041#comment-16351041 ] ASF GitHub Bot commented on METRON-1442: Github user cestella commented on the issue: https://github.com/apache/metron/pull/923 Looks like you didn't quite get all of the mock test infrastructure for `metron-rest` set up properly. I went ahead and submitted a PR against your branch to help out. :) If you can merge https://github.com/MohanDV/metron/pull/1 into your branch, it should fix the tests. > Rest endpoints for operations on Indexing topology is broken after splitting > the indexing topology > -- > > Key: METRON-1442 > URL: https://issues.apache.org/jira/browse/METRON-1442 > Project: Metron > Issue Type: Bug >Reporter: Mohan >Assignee: Mohan >Priority: Major > > Rest endpoints for operations on Indexing topology is broken after splitting > the indexing topology into random_access_indexing and batch_indexing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1441) Create complementary Solr schemas for the main sensors
[ https://issues.apache.org/jira/browse/METRON-1441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16351037#comment-16351037 ] ASF GitHub Bot commented on METRON-1441: Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/922 I tested this in full dev using the install script in https://github.com/apache/metron/pull/918. I was able to create collections for each schema except for "error". For that to work properly, I had to: - remove `docValues="true"` from the "bytes" field type - add the "guid" field used in other schemas Still working on indexing data into these collections but so far so good. > Create complementary Solr schemas for the main sensors > -- > > Key: METRON-1441 > URL: https://issues.apache.org/jira/browse/METRON-1441 > Project: Metron > Issue Type: Improvement >Reporter: Casey Stella >Priority: Major > > We have ES templates for bro, snort, yaf, and error, we need corresponding > solr schemas for these collections. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1443) Missing Critical MPack Install Instruction for Ubuntu
[ https://issues.apache.org/jira/browse/METRON-1443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350999#comment-16350999 ] ASF GitHub Bot commented on METRON-1443: GitHub user nickwallen opened a pull request: https://github.com/apache/metron/pull/925 METRON-1443 Missing Critical MPack Install Instruction for Ubuntu When installing Elasticsearch with the MPack on Ubuntu, you must manually install the Elasticsearch repositories. The Mpack itself does not do this, like it does on CentOS. When the development environment on Ubuntu is spun-up this step is performed within Ansible as a prerequisite to the Mpack install. Until this can be fixed so that it matches what happens on CentOS, this needs to be at least documented. I should have documented this in #903 , but did not do so. Oops. You can merge this pull request into a Git repository by running: $ git pull https://github.com/nickwallen/metron METRON-1443 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/metron/pull/925.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #925 commit 0c40178494d2a12e8e4abbd43ba4f85338aa05da Author: Nick Allen Date: 2018-02-02T22:12:23Z METRON-1443 Missing Critical MPack Install Instruction for Ubuntu > Missing Critical MPack Install Instruction for Ubuntu > - > > Key: METRON-1443 > URL: https://issues.apache.org/jira/browse/METRON-1443 > Project: Metron > Issue Type: Bug >Reporter: Nick Allen >Assignee: Nick Allen >Priority: Major > > When installing Elasticsearch with the MPack on Ubuntu, you must manually > install the Elasticsearch repositories. The Mpack itself does not do this, > like it does on CentOS. > When the development environment on Ubuntu is spun-up this step is performed > within Ansible as a prerequisite to the Mpack install. Until this can be > fixed so that it matches what happens in CentOS this needs to be at least > documented. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (METRON-1443) Missing Critical MPack Install Instruction for Ubuntu
Nick Allen created METRON-1443: -- Summary: Missing Critical MPack Install Instruction for Ubuntu Key: METRON-1443 URL: https://issues.apache.org/jira/browse/METRON-1443 Project: Metron Issue Type: Bug Reporter: Nick Allen Assignee: Nick Allen When installing Elasticsearch with the MPack on Ubuntu, you must manually install the Elasticsearch repositories. The Mpack itself does not do this, like it does on CentOS. When the development environment on Ubuntu is spun-up this step is performed within Ansible as a prerequisite to the Mpack install. Until this can be fixed so that it matches what happens in CentOS this needs to be at least documented. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1299) MetronError tests fail if hostname isn't set
[ https://issues.apache.org/jira/browse/METRON-1299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350995#comment-16350995 ] ASF GitHub Bot commented on METRON-1299: GitHub user ottobackwards opened a pull request: https://github.com/apache/metron/pull/924 METRON-1299 In MetronError tests, don't test for HostName if getHostName wouldn't work MetronError ignores exceptions from InetAddress.getLocalHost().getHostName() and leaves the field unset. The unit test however assumes it would be set, and someone has logged a jira on this, since it makes the build fail. Changed the test so that it only verifies the hostName if it would have worked. ### Testing - Code review - Tests Pass > no non-test changes in pr ```java private void addHostname(JSONObject errorMessage) { try { errorMessage.put(ErrorFields.HOSTNAME.getName(), InetAddress.getLocalHost().getHostName()); } catch (UnknownHostException ex) { // Leave the hostname field off if it cannot be found } } ``` ### For all changes: - [x] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel). - [x] Does your PR title start with METRON- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? You can merge this pull request into a Git repository by running: $ git pull https://github.com/ottobackwards/metron error_addHost Alternatively you can review and apply these changes as the patch at: https://github.com/apache/metron/pull/924.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #924 commit cee3acba914f97ca7d2faf6e7822c97928a4e242 Author: Otto Fowler Date: 2018-02-02T21:57:47Z do not test for hostName if calling hostName throws, since it will be null > MetronError tests fail if hostname isn't set > > > Key: METRON-1299 > URL: https://issues.apache.org/jira/browse/METRON-1299 > Project: Metron > Issue Type: Bug >Affects Versions: 0.4.1 > Environment: openSUSE Tumbleweed 20171102, OpenJDK 1.8.0_144 >Reporter: Stuart Bertram >Assignee: Otto Fowler >Priority: Major > > If I run "mvn package" in the root Metron directory then compilation fails > because of a null reference in > {{./metron-platform/metron-common/src/test/java/org/apache/metron/common/error/MetronErrorTest.java}}. > This happens in {{getJSONObjectShouldReturnBasicInformation}} on line 56 > because {{errorJSON.get(Constants.ErrorFields.HOSTNAME.getName())}} is > assumed to return a string and the {{length()}} method is called on it. > Because the assert doesn't use a message then no obvious reason why it fails > is logged. > The underlying problem is that > {{metron-platform/metron-common/src/main/java/org/apache/metron/common/error/MetronError.java}} > falls through to a {{catch}} block in {{addHostname()}} when > {{netAddress.getLocalHost().getHostName()}} exceptions with {{Name or service > not known}} for the host name. > Setting a hostname that resolves is a build requirement that is out of > Metron's control, but if the code specifically handles the fact that it won't > always be retrieved then it seems problematic to have tests that assume it > works _and_ not make it clear what the failure is when it occurs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (METRON-1299) MetronError tests fail if hostname isn't set
[ https://issues.apache.org/jira/browse/METRON-1299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Otto Fowler reassigned METRON-1299: --- Assignee: Otto Fowler (was: Mohan) > MetronError tests fail if hostname isn't set > > > Key: METRON-1299 > URL: https://issues.apache.org/jira/browse/METRON-1299 > Project: Metron > Issue Type: Bug >Affects Versions: 0.4.1 > Environment: openSUSE Tumbleweed 20171102, OpenJDK 1.8.0_144 >Reporter: Stuart Bertram >Assignee: Otto Fowler >Priority: Major > > If I run "mvn package" in the root Metron directory then compilation fails > because of a null reference in > {{./metron-platform/metron-common/src/test/java/org/apache/metron/common/error/MetronErrorTest.java}}. > This happens in {{getJSONObjectShouldReturnBasicInformation}} on line 56 > because {{errorJSON.get(Constants.ErrorFields.HOSTNAME.getName())}} is > assumed to return a string and the {{length()}} method is called on it. > Because the assert doesn't use a message then no obvious reason why it fails > is logged. > The underlying problem is that > {{metron-platform/metron-common/src/main/java/org/apache/metron/common/error/MetronError.java}} > falls through to a {{catch}} block in {{addHostname()}} when > {{netAddress.getLocalHost().getHostName()}} exceptions with {{Name or service > not known}} for the host name. > Setting a hostname that resolves is a build requirement that is out of > Metron's control, but if the code specifically handles the fact that it won't > always be retrieved then it seems problematic to have tests that assume it > works _and_ not make it clear what the failure is when it occurs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1441) Create complementary Solr schemas for the main sensors
[ https://issues.apache.org/jira/browse/METRON-1441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350983#comment-16350983 ] ASF GitHub Bot commented on METRON-1441: Github user cestella commented on the issue: https://github.com/apache/metron/pull/922 Ok, I did the following: * Augmented the readme to point to the Solr documentation around schemas. Keep in mind, this is intermediate work that will feed into the "install Solr" work * Added yaf and error schemas * Renamed the test to an integration test * Moved the data from multiline to separate files > Create complementary Solr schemas for the main sensors > -- > > Key: METRON-1441 > URL: https://issues.apache.org/jira/browse/METRON-1441 > Project: Metron > Issue Type: Improvement >Reporter: Casey Stella >Priority: Major > > We have ES templates for bro, snort, yaf, and error, we need corresponding > solr schemas for these collections. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (METRON-1230) As a stopgap prior to METRON-777, add more simplistic sideloading of custom Parsers
[ https://issues.apache.org/jira/browse/METRON-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Otto Fowler reassigned METRON-1230: --- Assignee: Casey Stella > As a stopgap prior to METRON-777, add more simplistic sideloading of custom > Parsers > --- > > Key: METRON-1230 > URL: https://issues.apache.org/jira/browse/METRON-1230 > Project: Metron > Issue Type: Improvement >Reporter: Casey Stella >Assignee: Casey Stella >Priority: Major > > Until we get 777 in, it'd be nice to have a simple ability using the normal > storm functionality to enable users to provide custom parsers without forking > Metron. This should be done via simply creating a jar with their code (and > bundled dependencies) and have it picked up and available to the REST Service > (and consequently the management UI) as well as the start_parser_topology.sh > command. > This should be minimal movement as we have a more robust solution coming with > METRON-777. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (METRON-1302) Split up Indexing Topology into batch and random access sections
[ https://issues.apache.org/jira/browse/METRON-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Otto Fowler reassigned METRON-1302: --- Assignee: Casey Stella > Split up Indexing Topology into batch and random access sections > > > Key: METRON-1302 > URL: https://issues.apache.org/jira/browse/METRON-1302 > Project: Metron > Issue Type: Improvement >Reporter: Casey Stella >Assignee: Casey Stella >Priority: Major > > Currently we have the indexing topology handle writing to both random access > indices (e.g. elasticsearch) as well as batch write indices (e.g. hdfs). We > should split these up and configure them separately. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-809) Error while running metron_full_install.yml
[ https://issues.apache.org/jira/browse/METRON-809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350975#comment-16350975 ] Otto Fowler commented on METRON-809: Closing inactive > Error while running metron_full_install.yml > --- > > Key: METRON-809 > URL: https://issues.apache.org/jira/browse/METRON-809 > Project: Metron > Issue Type: Bug > Environment: Centos 6 >Reporter: shriram >Priority: Blocker > Attachments: Capture.JPG > > > I am trying to install Metron cluster in AWS with CentOS 6. I encountered the > following error while running metron_full_install.yml, at its last step: > "[ERROR] Failed to execute goal > org.codehaus.mojo:exec-maven-plugin:1.5.0:exec (docker-build) on project > metron-rpm: Command execution failed. Cannot run program \"docker\" (in > directory > \"/opt/incubator-metron/metron-deployment/packaging/docker/rpm-docker\"): > error=2, No such file or directory -> [Help 1]", "[ERROR] ", "[ERROR] To see > the full stack trace of the errors, re-run Maven with the -e switch.", > "[ERROR] Re-run Maven using the -X switch to enable full debug logging.", > "[ERROR] ", "[ERROR] For more information about the errors and possible > solutions, please read the following articles:", "[ERROR] [Help 1] > http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException";, > "[ERROR] ", "[ERROR] After correcting the problems, you can resume the build > with the command", "[ERROR] mvn -rf :metron-rpm"], "warnings": []} > Attaching screenshot for more details. > Thanks. > Regards, > Shriram -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1437) STELLAR: New Shell function to make a list from some number of variables
[ https://issues.apache.org/jira/browse/METRON-1437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350973#comment-16350973 ] Otto Fowler commented on METRON-1437: - I'm not to bright. "[var, var2, var4]" > STELLAR: New Shell function to make a list from some number of variables > > > Key: METRON-1437 > URL: https://issues.apache.org/jira/browse/METRON-1437 > Project: Metron > Issue Type: New Feature >Reporter: Otto Fowler >Assignee: Otto Fowler >Priority: Major > > The ability in the shell to make a list of a group of variables, the idea > being you can then pass that list to other functions -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1212) Bundles and Maven Plugin
[ https://issues.apache.org/jira/browse/METRON-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350940#comment-16350940 ] ASF GitHub Bot commented on METRON-1212: Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/865 Hey @JonZeolla how is it going? > Bundles and Maven Plugin > > > Key: METRON-1212 > URL: https://issues.apache.org/jira/browse/METRON-1212 > Project: Metron > Issue Type: Sub-task >Reporter: Otto Fowler >Assignee: Otto Fowler >Priority: Major > Labels: metron-feature-canidate, > metron-feature-extensions-parsers > > The first effort will be to land the bundle system and supporting maven > plugin on master -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1442) Rest endpoints for operations on Indexing topology is broken after splitting the indexing topology
[ https://issues.apache.org/jira/browse/METRON-1442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350926#comment-16350926 ] ASF GitHub Bot commented on METRON-1442: GitHub user MohanDV opened a pull request: https://github.com/apache/metron/pull/923 METRON-1442: Split rest end points for indexing topology into random access indexing and batch indexing ## Contributor Comments Split rest end points for indexing topology into random access indexing topology and batch indexing topology to support the start/stop/activate/deactivate/status operations on the respective topologies. **Steps to Verify** 1 . Spin up Full Dev 2. Go to Swagger at http://node1:8082/swagger-ui.html#!/storm-controller/ 3. You should see the below newly added rest endpoints https://user-images.githubusercontent.com/12934693/35753803-1f96b6c4-0887-11e8-9d55-6e185e34cc52.png";> ## Pull Request Checklist Thank you for submitting a contribution to Apache Metron. Please refer to our [Development Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235) for the complete guide to follow for contributions. Please refer also to our [Build Verification Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview) for complete smoke testing guides. In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel). - [x] Does your PR title start with METRON- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? ### For code changes: - [x] Have you included steps to reproduce the behavior or problem that is being changed or addressed? - [x] Have you included steps or a guide to how the change may be verified and tested manually? - [ ] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via: ``` mvn -q clean integration-test install && dev-utilities/build-utils/verify_licenses.sh ``` - [x] Have you written or updated unit tests and or integration tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`: ``` cd site-book mvn site ``` Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. It is also recommended that [travis-ci](https://travis-ci.org) is set up for your personal repository such that your branches are built there before submitting a pull request. You can merge this pull request into a Git repository by running: $ git pull https://github.com/MohanDV/metron METRON-1442 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/metron/pull/923.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #923 commit 59a5f26195032c33e0cf56a0abf0153cda09d71d Author: Mohan Venkateshaiah Date: 2018-02-02T20:24:34Z Splitted rest end points for indexing topology to support the start/stop/activate/deactivate/status operations on the randomaccess and batch indexing topology > Rest endpoints for operations on Indexing topology is broken after splitting > the indexing topology > -- > > Key: METRON-1442 > URL: https://issues.apache.org/jira/browse/METRON-1442 > Project: Metron > Issue Type: Bug >Reporter: Mohan >Assignee: Mohan >Priority: Major > > Rest endpoints for operations on Indexing topology is broken after splitting > the ind
[jira] [Commented] (METRON-1441) Create complementary Solr schemas for the main sensors
[ https://issues.apache.org/jira/browse/METRON-1441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350714#comment-16350714 ] ASF GitHub Bot commented on METRON-1441: Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/922#discussion_r165711148 --- Diff: metron-platform/metron-solr/src/test/java/org/apache/metron/solr/schema/SchemaTranslatorTest.java --- @@ -0,0 +1,188 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.metron.solr.schema; + +import com.google.common.base.Splitter; +import com.google.common.collect.Iterables; +import org.adrianwalker.multilinestring.Multiline; +import org.apache.metron.common.configuration.writer.WriterConfiguration; +import org.apache.metron.common.utils.JSONUtils; +import org.apache.metron.integration.UnableToStartException; +import org.apache.metron.solr.integration.components.SolrComponent; +import org.apache.metron.solr.writer.SolrWriter; +import org.json.simple.JSONObject; +import org.junit.Assert; +import org.junit.Test; + +import java.io.IOException; +import java.io.PrintWriter; +import java.io.StringWriter; +import java.util.*; + +public class SchemaTranslatorTest { + + /** +{"adapter.threatinteladapter.end.ts":"1517499201357","bro_timestamp":"1517499194.7338","ip_dst_port":8080,"enrichmentsplitterbolt.splitter.end.ts":"1517499201202","enrichmentsplitterbolt.splitter.begin.ts":"1517499201200","adapter.hostfromjsonlistadapter.end.ts":"1517499201207","adapter.geoadapter.begin.ts":"1517499201209","uid":"CUrRne3iLIxXavQtci","trans_depth":143,"protocol":"http","original_string":"HTTP | id.orig_p:50451 method:GET request_body_len:0 id.resp_p:8080 uri:\/api\/v1\/clusters\/metron_cluster\/services\/KAFKA\/components\/KAFKA_BROKER?fields=metrics\/kafka\/server\/BrokerTopicMetrics\/AllTopicsBytesInPerSec\/1MinuteRate[1484165330,1484168930,15],metrics\/kafka\/server\/BrokerTopicMetrics\/AllTopicsBytesOutPerSec\/1MinuteRate[1484165330,1484168930,15],metrics\/kafka\/server\/BrokerTopicMetrics\/AllTopicsMessagesInPerSec\/1MinuteRate[1484165330,1484168930,15],metrics\/kafka\/controller\/KafkaController\/ActiveControllerCount[1484165330,1484168930,15],metrics\/kafka\/controller\/ControllerStats\/LeaderElectionRateAndTimeMs\/1MinuteRate[1484165330,1484168930,15],metrics\/kafka\/controller\/ControllerStats\/UncleanLeaderElectionsPerSec\/1MinuteRate[1484165330,1484168930,15],metrics\/kafka\/server\/ReplicaFetcherManager\/Replica-MaxLag[1484165330,1484168930,15],metrics\/kafka\/server\/ReplicaManager\/PartitionCount[1484165330,1484168930,15],metrics\/kafka\/server\/ReplicaManager\/UnderReplicatedPartitions[1484165330,1484168930,15],metrics\/kafka\/server\/ReplicaManager\/LeaderCount[1484165330,1484168930,15]&format=null_padding&_=1484168930776 tags:[] uid:CUrRne3iLIxXavQtci referrer:http:\/\/node1:8080\/ trans_depth:143 host:node1 id.orig_h:192.168.66.1 response_body_len:0 user_agent:Mozilla\/5.0 (Macintosh; Intel Mac OS X 10_12_2) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/55.0.2883.95 Safari\/537.36 ts:1517499194.7338 id.resp_h:192.168.66.121","ip_dst_addr":"192.168.66.121","threatinteljoinbolt.joiner.ts":"1517499201359","host":"node1","enrichmentjoinbolt.joiner.ts":"1517499201212","adapter.hostfromjsonlistadapter.begin.ts":"1517499201206","threatintelsplitterbolt.splitter.begin.ts":"1517499201215","ip_src_addr":"192.168.66.1","user_agent":"Mozilla\/5.0 (Macintosh; Intel Mac OS X 10_12_2) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/55.0.2883.95 Safari\/537.36","timestamp":1517499194733,"method":"GET","request_body_len":0,"uri":"\/api\/v1\/clusters\/metron_cluster\/services\/KAFKA\/components\/KAFKA_BROKER?fields=metrics\/kafka\/server\/BrokerTopicMetrics\/AllTopicsBytesInPerSec\/1MinuteRate[1484165330,1484168930,15],metrics\/kafka\/server\/BrokerTopicMetrics\/AllTopicsBytesOutPerSec\/1MinuteRate[1484165330,1484168930,15],metrics\/kafka\/server\/BrokerTopicMetrics\/AllTo
[jira] [Commented] (METRON-1438) STELLAR: Move shell functions to common from metron-management
[ https://issues.apache.org/jira/browse/METRON-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350575#comment-16350575 ] ASF GitHub Bot commented on METRON-1438: Github user asfgit closed the pull request at: https://github.com/apache/metron/pull/920 > STELLAR: Move shell functions to common from metron-management > -- > > Key: METRON-1438 > URL: https://issues.apache.org/jira/browse/METRON-1438 > Project: Metron > Issue Type: Sub-task >Reporter: Otto Fowler >Assignee: Otto Fowler >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1436) Manually Install Solr Cloud in Full Dev
[ https://issues.apache.org/jira/browse/METRON-1436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350547#comment-16350547 ] ASF GitHub Bot commented on METRON-1436: Github user mmiklavc closed the pull request at: https://github.com/apache/metron/pull/918 > Manually Install Solr Cloud in Full Dev > --- > > Key: METRON-1436 > URL: https://issues.apache.org/jira/browse/METRON-1436 > Project: Metron > Issue Type: Sub-task >Reporter: Michael Miklavcic >Assignee: Michael Miklavcic >Priority: Major > > Script and documentation for getting Solr setup in full dev -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1438) STELLAR: Move shell functions to common from metron-management
[ https://issues.apache.org/jira/browse/METRON-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350434#comment-16350434 ] ASF GitHub Bot commented on METRON-1438: Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/920#discussion_r165663581 --- Diff: metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/ShellFunctionsTest.java --- @@ -40,8 +45,8 @@ ); Context context = new Context.Builder() -.with(Context.Capabilities.SHELL_VARIABLES , () -> variables) -.build(); +.with(Context.Capabilities.SHELL_VARIABLES , () -> variables).build(); --- End diff -- Right now, I never let my IDE reformat for me. Like you said, if we get the code base matching check style and I can load that style into my IDE, then I'd gladly let it do most of the work for me. Maybe I'll open a discuss thread. I don't know how to handle this kind of thing and it happens all the time. But for this specific scenario in your PR, it really doesn't matter either way. I think you're good to go either way. > STELLAR: Move shell functions to common from metron-management > -- > > Key: METRON-1438 > URL: https://issues.apache.org/jira/browse/METRON-1438 > Project: Metron > Issue Type: Sub-task >Reporter: Otto Fowler >Assignee: Otto Fowler >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1438) STELLAR: Move shell functions to common from metron-management
[ https://issues.apache.org/jira/browse/METRON-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350435#comment-16350435 ] ASF GitHub Bot commented on METRON-1438: Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/920 +1 Thanks @ottobackwards > STELLAR: Move shell functions to common from metron-management > -- > > Key: METRON-1438 > URL: https://issues.apache.org/jira/browse/METRON-1438 > Project: Metron > Issue Type: Sub-task >Reporter: Otto Fowler >Assignee: Otto Fowler >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1441) Create complementary Solr schemas for the main sensors
[ https://issues.apache.org/jira/browse/METRON-1441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350427#comment-16350427 ] ASF GitHub Bot commented on METRON-1441: Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/922#discussion_r165662614 --- Diff: metron-platform/metron-solr/src/test/java/org/apache/metron/solr/schema/SchemaTranslatorTest.java --- @@ -0,0 +1,188 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.metron.solr.schema; + +import com.google.common.base.Splitter; +import com.google.common.collect.Iterables; +import org.adrianwalker.multilinestring.Multiline; +import org.apache.metron.common.configuration.writer.WriterConfiguration; +import org.apache.metron.common.utils.JSONUtils; +import org.apache.metron.integration.UnableToStartException; +import org.apache.metron.solr.integration.components.SolrComponent; +import org.apache.metron.solr.writer.SolrWriter; +import org.json.simple.JSONObject; +import org.junit.Assert; +import org.junit.Test; + +import java.io.IOException; +import java.io.PrintWriter; +import java.io.StringWriter; +import java.util.*; + +public class SchemaTranslatorTest { + + /** +{"adapter.threatinteladapter.end.ts":"1517499201357","bro_timestamp":"1517499194.7338","ip_dst_port":8080,"enrichmentsplitterbolt.splitter.end.ts":"1517499201202","enrichmentsplitterbolt.splitter.begin.ts":"1517499201200","adapter.hostfromjsonlistadapter.end.ts":"1517499201207","adapter.geoadapter.begin.ts":"1517499201209","uid":"CUrRne3iLIxXavQtci","trans_depth":143,"protocol":"http","original_string":"HTTP | id.orig_p:50451 method:GET request_body_len:0 id.resp_p:8080 uri:\/api\/v1\/clusters\/metron_cluster\/services\/KAFKA\/components\/KAFKA_BROKER?fields=metrics\/kafka\/server\/BrokerTopicMetrics\/AllTopicsBytesInPerSec\/1MinuteRate[1484165330,1484168930,15],metrics\/kafka\/server\/BrokerTopicMetrics\/AllTopicsBytesOutPerSec\/1MinuteRate[1484165330,1484168930,15],metrics\/kafka\/server\/BrokerTopicMetrics\/AllTopicsMessagesInPerSec\/1MinuteRate[1484165330,1484168930,15],metrics\/kafka\/controller\/KafkaController\/ActiveControllerCount[1484165330,1484168930,15],metrics\/kafka\/controller\/ControllerStats\/LeaderElectionRateAndTimeMs\/1MinuteRate[1484165330,1484168930,15],metrics\/kafka\/controller\/ControllerStats\/UncleanLeaderElectionsPerSec\/1MinuteRate[1484165330,1484168930,15],metrics\/kafka\/server\/ReplicaFetcherManager\/Replica-MaxLag[1484165330,1484168930,15],metrics\/kafka\/server\/ReplicaManager\/PartitionCount[1484165330,1484168930,15],metrics\/kafka\/server\/ReplicaManager\/UnderReplicatedPartitions[1484165330,1484168930,15],metrics\/kafka\/server\/ReplicaManager\/LeaderCount[1484165330,1484168930,15]&format=null_padding&_=1484168930776 tags:[] uid:CUrRne3iLIxXavQtci referrer:http:\/\/node1:8080\/ trans_depth:143 host:node1 id.orig_h:192.168.66.1 response_body_len:0 user_agent:Mozilla\/5.0 (Macintosh; Intel Mac OS X 10_12_2) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/55.0.2883.95 Safari\/537.36 ts:1517499194.7338 id.resp_h:192.168.66.121","ip_dst_addr":"192.168.66.121","threatinteljoinbolt.joiner.ts":"1517499201359","host":"node1","enrichmentjoinbolt.joiner.ts":"1517499201212","adapter.hostfromjsonlistadapter.begin.ts":"1517499201206","threatintelsplitterbolt.splitter.begin.ts":"1517499201215","ip_src_addr":"192.168.66.1","user_agent":"Mozilla\/5.0 (Macintosh; Intel Mac OS X 10_12_2) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/55.0.2883.95 Safari\/537.36","timestamp":1517499194733,"method":"GET","request_body_len":0,"uri":"\/api\/v1\/clusters\/metron_cluster\/services\/KAFKA\/components\/KAFKA_BROKER?fields=metrics\/kafka\/server\/BrokerTopicMetrics\/AllTopicsBytesInPerSec\/1MinuteRate[1484165330,1484168930,15],metrics\/kafka\/server\/BrokerTopicMetrics\/AllTopicsBytesOutPerSec\/1MinuteRate[1484165330,1484168930,15],metrics\/kafka\/server\/BrokerTopicMetrics\/
[jira] [Commented] (METRON-1438) STELLAR: Move shell functions to common from metron-management
[ https://issues.apache.org/jira/browse/METRON-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350409#comment-16350409 ] ASF GitHub Bot commented on METRON-1438: Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/920#discussion_r165660358 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/shell/cli/PausableInput.java --- @@ -36,8 +37,8 @@ * */ public class PausableInput extends InputStream { - InputStream in = System.in; - boolean paused = false; + private InputStream in = System.in; + private AtomicBoolean paused = new AtomicBoolean(false); --- End diff -- Good find! I'm sure that was frustrating to dig into. > STELLAR: Move shell functions to common from metron-management > -- > > Key: METRON-1438 > URL: https://issues.apache.org/jira/browse/METRON-1438 > Project: Metron > Issue Type: Sub-task >Reporter: Otto Fowler >Assignee: Otto Fowler >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1438) STELLAR: Move shell functions to common from metron-management
[ https://issues.apache.org/jira/browse/METRON-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350422#comment-16350422 ] ASF GitHub Bot commented on METRON-1438: Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/920#discussion_r165662017 --- Diff: metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/ShellFunctionsTest.java --- @@ -40,8 +45,8 @@ ); Context context = new Context.Builder() -.with(Context.Capabilities.SHELL_VARIABLES , () -> variables) -.build(); +.with(Context.Capabilities.SHELL_VARIABLES , () -> variables).build(); --- End diff -- How do you have your formatting preferences set to get the above? > STELLAR: Move shell functions to common from metron-management > -- > > Key: METRON-1438 > URL: https://issues.apache.org/jira/browse/METRON-1438 > Project: Metron > Issue Type: Sub-task >Reporter: Otto Fowler >Assignee: Otto Fowler >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1438) STELLAR: Move shell functions to common from metron-management
[ https://issues.apache.org/jira/browse/METRON-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350419#comment-16350419 ] ASF GitHub Bot commented on METRON-1438: Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/920#discussion_r165661676 --- Diff: metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/ShellFunctionsTest.java --- @@ -40,8 +45,8 @@ ); Context context = new Context.Builder() -.with(Context.Capabilities.SHELL_VARIABLES , () -> variables) -.build(); +.with(Context.Capabilities.SHELL_VARIABLES , () -> variables).build(); --- End diff -- I didn't reformat that like that on purpose. I had a period where I was setting the CONSOLE capability. When I removed it, it just worked out like this. Even if I select and format in intellij it doesn't change the .build() to the next line. I think what you have above is fine. I would like it to 'just' happen if I format code though, since it is easy for things to slip through. It is tough right now, because so much of the codebase isn't formatted to check style, and I don't think we want every pr to include a lot of formatting changes. > STELLAR: Move shell functions to common from metron-management > -- > > Key: METRON-1438 > URL: https://issues.apache.org/jira/browse/METRON-1438 > Project: Metron > Issue Type: Sub-task >Reporter: Otto Fowler >Assignee: Otto Fowler >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1435) Management UI cannot save json objects in advanced config
[ https://issues.apache.org/jira/browse/METRON-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350413#comment-16350413 ] ASF GitHub Bot commented on METRON-1435: Github user asfgit closed the pull request at: https://github.com/apache/metron/pull/917 > Management UI cannot save json objects in advanced config > - > > Key: METRON-1435 > URL: https://issues.apache.org/jira/browse/METRON-1435 > Project: Metron > Issue Type: Bug >Reporter: Ryan Merriman >Assignee: Ryan Merriman >Priority: Major > > Currently any value entered into the Advanced Config section of the > Management UI is converted to string. It should allow arrays or json objects > to be saved as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1438) STELLAR: Move shell functions to common from metron-management
[ https://issues.apache.org/jira/browse/METRON-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350404#comment-16350404 ] ASF GitHub Bot commented on METRON-1438: Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/920#discussion_r165659854 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/shell/cli/PausableInput.java --- @@ -36,8 +37,8 @@ * */ public class PausableInput extends InputStream { - InputStream in = System.in; - boolean paused = false; + private InputStream in = System.in; + private AtomicBoolean paused = new AtomicBoolean(false); --- End diff -- I found in travis and locally, that the PausableInput was hanging, my builds where not completing. PausableInput.INSTANCE.unpause(); was never returning. I couldn't figure out why it was locked. Reading the class documentation and the trying to understand the threading in the shell, I looked at the class and saw some inconsistencies with how and when we checked if we where paused in the different read() calls, and also that the flag field was not volatile or atomic. I refactored this things to what I *think* would be the correct and consistent approaches and it resolved my issue. > STELLAR: Move shell functions to common from metron-management > -- > > Key: METRON-1438 > URL: https://issues.apache.org/jira/browse/METRON-1438 > Project: Metron > Issue Type: Sub-task >Reporter: Otto Fowler >Assignee: Otto Fowler >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1441) Create complementary Solr schemas for the main sensors
[ https://issues.apache.org/jira/browse/METRON-1441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350386#comment-16350386 ] ASF GitHub Bot commented on METRON-1441: Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/922#discussion_r165656511 --- Diff: metron-platform/metron-solr/src/test/java/org/apache/metron/solr/schema/SchemaTranslatorTest.java --- @@ -0,0 +1,188 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.metron.solr.schema; + +import com.google.common.base.Splitter; +import com.google.common.collect.Iterables; +import org.adrianwalker.multilinestring.Multiline; +import org.apache.metron.common.configuration.writer.WriterConfiguration; +import org.apache.metron.common.utils.JSONUtils; +import org.apache.metron.integration.UnableToStartException; +import org.apache.metron.solr.integration.components.SolrComponent; +import org.apache.metron.solr.writer.SolrWriter; +import org.json.simple.JSONObject; +import org.junit.Assert; +import org.junit.Test; + +import java.io.IOException; +import java.io.PrintWriter; +import java.io.StringWriter; +import java.util.*; + --- End diff -- Yes, I absolutely can. > Create complementary Solr schemas for the main sensors > -- > > Key: METRON-1441 > URL: https://issues.apache.org/jira/browse/METRON-1441 > Project: Metron > Issue Type: Improvement >Reporter: Casey Stella >Priority: Major > > We have ES templates for bro, snort, yaf, and error, we need corresponding > solr schemas for these collections. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1436) Manually Install Solr Cloud in Full Dev
[ https://issues.apache.org/jira/browse/METRON-1436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350393#comment-16350393 ] ASF GitHub Bot commented on METRON-1436: Github user justinleet commented on the issue: https://github.com/apache/metron/pull/918 Thanks for the updates. I'm +1 on including this in the feature branch. > Manually Install Solr Cloud in Full Dev > --- > > Key: METRON-1436 > URL: https://issues.apache.org/jira/browse/METRON-1436 > Project: Metron > Issue Type: Sub-task >Reporter: Michael Miklavcic >Assignee: Michael Miklavcic >Priority: Major > > Script and documentation for getting Solr setup in full dev -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1441) Create complementary Solr schemas for the main sensors
[ https://issues.apache.org/jira/browse/METRON-1441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350392#comment-16350392 ] ASF GitHub Bot commented on METRON-1441: Github user cestella commented on the issue: https://github.com/apache/metron/pull/922 @ottobackwards Very likely these schema files won't stay in this spot, but the final resting spot won't be apparent until we figure out how to automatically apply the schemas. Treat this PR as just unlocking progress for downstream PRs (like correcting SolrWriter to write to Solr again). > Create complementary Solr schemas for the main sensors > -- > > Key: METRON-1441 > URL: https://issues.apache.org/jira/browse/METRON-1441 > Project: Metron > Issue Type: Improvement >Reporter: Casey Stella >Priority: Major > > We have ES templates for bro, snort, yaf, and error, we need corresponding > solr schemas for these collections. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1441) Create complementary Solr schemas for the main sensors
[ https://issues.apache.org/jira/browse/METRON-1441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350387#comment-16350387 ] ASF GitHub Bot commented on METRON-1441: Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/922#discussion_r165656636 --- Diff: metron-platform/metron-solr/src/test/java/org/apache/metron/solr/schema/SchemaTranslatorTest.java --- @@ -0,0 +1,188 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.metron.solr.schema; + +import com.google.common.base.Splitter; +import com.google.common.collect.Iterables; +import org.adrianwalker.multilinestring.Multiline; +import org.apache.metron.common.configuration.writer.WriterConfiguration; +import org.apache.metron.common.utils.JSONUtils; +import org.apache.metron.integration.UnableToStartException; +import org.apache.metron.solr.integration.components.SolrComponent; +import org.apache.metron.solr.writer.SolrWriter; +import org.json.simple.JSONObject; +import org.junit.Assert; +import org.junit.Test; + +import java.io.IOException; +import java.io.PrintWriter; +import java.io.StringWriter; +import java.util.*; + +public class SchemaTranslatorTest { + + /** +{"adapter.threatinteladapter.end.ts":"1517499201357","bro_timestamp":"1517499194.7338","ip_dst_port":8080,"enrichmentsplitterbolt.splitter.end.ts":"1517499201202","enrichmentsplitterbolt.splitter.begin.ts":"1517499201200","adapter.hostfromjsonlistadapter.end.ts":"1517499201207","adapter.geoadapter.begin.ts":"1517499201209","uid":"CUrRne3iLIxXavQtci","trans_depth":143,"protocol":"http","original_string":"HTTP | id.orig_p:50451 method:GET request_body_len:0 id.resp_p:8080 uri:\/api\/v1\/clusters\/metron_cluster\/services\/KAFKA\/components\/KAFKA_BROKER?fields=metrics\/kafka\/server\/BrokerTopicMetrics\/AllTopicsBytesInPerSec\/1MinuteRate[1484165330,1484168930,15],metrics\/kafka\/server\/BrokerTopicMetrics\/AllTopicsBytesOutPerSec\/1MinuteRate[1484165330,1484168930,15],metrics\/kafka\/server\/BrokerTopicMetrics\/AllTopicsMessagesInPerSec\/1MinuteRate[1484165330,1484168930,15],metrics\/kafka\/controller\/KafkaController\/ActiveControllerCount[1484165330,1484168930,15],metrics\/kafka\/controller\/ControllerStats\/LeaderElectionRateAndTimeMs\/1MinuteRate[1484165330,1484168930,15],metrics\/kafka\/controller\/ControllerStats\/UncleanLeaderElectionsPerSec\/1MinuteRate[1484165330,1484168930,15],metrics\/kafka\/server\/ReplicaFetcherManager\/Replica-MaxLag[1484165330,1484168930,15],metrics\/kafka\/server\/ReplicaManager\/PartitionCount[1484165330,1484168930,15],metrics\/kafka\/server\/ReplicaManager\/UnderReplicatedPartitions[1484165330,1484168930,15],metrics\/kafka\/server\/ReplicaManager\/LeaderCount[1484165330,1484168930,15]&format=null_padding&_=1484168930776 tags:[] uid:CUrRne3iLIxXavQtci referrer:http:\/\/node1:8080\/ trans_depth:143 host:node1 id.orig_h:192.168.66.1 response_body_len:0 user_agent:Mozilla\/5.0 (Macintosh; Intel Mac OS X 10_12_2) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/55.0.2883.95 Safari\/537.36 ts:1517499194.7338 id.resp_h:192.168.66.121","ip_dst_addr":"192.168.66.121","threatinteljoinbolt.joiner.ts":"1517499201359","host":"node1","enrichmentjoinbolt.joiner.ts":"1517499201212","adapter.hostfromjsonlistadapter.begin.ts":"1517499201206","threatintelsplitterbolt.splitter.begin.ts":"1517499201215","ip_src_addr":"192.168.66.1","user_agent":"Mozilla\/5.0 (Macintosh; Intel Mac OS X 10_12_2) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/55.0.2883.95 Safari\/537.36","timestamp":1517499194733,"method":"GET","request_body_len":0,"uri":"\/api\/v1\/clusters\/metron_cluster\/services\/KAFKA\/components\/KAFKA_BROKER?fields=metrics\/kafka\/server\/BrokerTopicMetrics\/AllTopicsBytesInPerSec\/1MinuteRate[1484165330,1484168930,15],metrics\/kafka\/server\/BrokerTopicMetrics\/AllTopicsBytesOutPerSec\/1MinuteRate[1484165330,1484168930,15],metrics\/kafka\/server\/BrokerTopicMetrics\/AllTo
[jira] [Commented] (METRON-1438) STELLAR: Move shell functions to common from metron-management
[ https://issues.apache.org/jira/browse/METRON-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350380#comment-16350380 ] ASF GitHub Bot commented on METRON-1438: Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/920#discussion_r165654991 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/shell/cli/PausableInput.java --- @@ -36,8 +37,8 @@ * */ public class PausableInput extends InputStream { - InputStream in = System.in; - boolean paused = false; + private InputStream in = System.in; + private AtomicBoolean paused = new AtomicBoolean(false); --- End diff -- What problem were you solving here @ottobackwards ? Is this bit access by multiple threads? > STELLAR: Move shell functions to common from metron-management > -- > > Key: METRON-1438 > URL: https://issues.apache.org/jira/browse/METRON-1438 > Project: Metron > Issue Type: Sub-task >Reporter: Otto Fowler >Assignee: Otto Fowler >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1438) STELLAR: Move shell functions to common from metron-management
[ https://issues.apache.org/jira/browse/METRON-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350379#comment-16350379 ] ASF GitHub Bot commented on METRON-1438: Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/920#discussion_r165653517 --- Diff: metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/ShellFunctionsTest.java --- @@ -40,8 +45,8 @@ ); Context context = new Context.Builder() -.with(Context.Capabilities.SHELL_VARIABLES , () -> variables) -.build(); +.with(Context.Capabilities.SHELL_VARIABLES , () -> variables).build(); --- End diff -- I am actually interested in what direction as a project we should be taking with these types of fluent, chained statements. I run across this all the time and I want to know the 'right' way that I should be doing it for the project. IMHO, the way it was (separated by a line break) is more readable. Meaning, a long set of chained statements should be separated by line breaks. For example... ``` result = new ProfileMeasurement() .withProfileName(profileName) .withEntity(entity) .withGroups(groups) .withPeriod(period) .withProfileValue(profileValue) .withTriageValues(triageValues) .withDefinition(definition); ``` But, of course, in terms of code style my opinion doesn't matter. It is all about our style guidelines. What does the Google code style guidelines say? Doesn't [this](https://google.github.io/styleguide/javaguide.html#s4.5.1-line-wrapping-where-to-break) support what I have said above about line breaks in this case? > STELLAR: Move shell functions to common from metron-management > -- > > Key: METRON-1438 > URL: https://issues.apache.org/jira/browse/METRON-1438 > Project: Metron > Issue Type: Sub-task >Reporter: Otto Fowler >Assignee: Otto Fowler >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (METRON-1442) Rest endpoints for operations on Indexing topology is broken after splitting the indexing topology
Mohan created METRON-1442: - Summary: Rest endpoints for operations on Indexing topology is broken after splitting the indexing topology Key: METRON-1442 URL: https://issues.apache.org/jira/browse/METRON-1442 Project: Metron Issue Type: Bug Reporter: Mohan Assignee: Mohan Rest endpoints for operations on Indexing topology is broken after splitting the indexing topology into random_access_indexing and batch_indexing -- This message was sent by Atlassian JIRA (v7.6.3#76005)