[GitHub] metron pull request #788: METRON-1223: Support for adding comments to alerts
Github user asfgit closed the pull request at: https://github.com/apache/metron/pull/788 ---
[GitHub] metron pull request #799: METRON-1249 Improve Metron MPack service checks
GitHub user nickwallen opened a pull request: https://github.com/apache/metron/pull/799 METRON-1249 Improve Metron MPack service checks This PR enhances the Metron 'Service Check' functionality in the MPack. The Service Check is an easy way for a user to know if their Metron cluster is healthy. The Service Check can be run manually from within Ambari. It is also automatically executed after kerberization. In the current version of the Service Check, healthy means the Parser and Indexing topologies are running. This PR enhances that to validate all of the install actions that occur across each of the Metron services. These checks include the following. * Kafka topics, user permissions, group permissions * HBase tables, column families, and user permissions * HDFS resources like the grok patterns and geo database * Ensures all Metron topologies are running * Ensures the web-based resources are responding I added considerable logging so that if a check does fail, a user will have a reasonable chance to understand why. Ambari doesn't give me an easy way to tell a user "hey, this is the problem!", so the user still has to go through the output of the Service Check in the Operations Panel to know why the Service Check failed. ## Testing * I manually tested each of the checks by, for example, deleting a Kafka topic then running the Service Check. This can be repeated for all of the different types of checks that I outlined above. * I tested the Service Check on a fresh deployment of Full Dev. * I then kerberized Full Dev and again validated the Service Check ## Pull Request Checklist - [ ] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel). - [ ] Does your PR title start with METRON- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Have you included steps to reproduce the behavior or problem that is being changed or addressed? - [ ] Have you included steps or a guide to how the change may be verified and tested manually? - [ ] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via: - [ ] Have you written or updated unit tests and or integration tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent? You can merge this pull request into a Git repository by running: $ git pull https://github.com/nickwallen/metron METRON-1249 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/metron/pull/799.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #799 commit 9619bad0ae9b2afa203fc376fec1856009ca02db Author: Nick Allen Date: 2017-09-21T18:09:28Z METRON-1249 Improve Metron MPack service checks ---
[GitHub] metron pull request #710: Metron-1083: Add filters using faceted search capa...
Github user asfgit closed the pull request at: https://github.com/apache/metron/pull/710 ---
[GitHub] metron issue #710: Metron-1083: Add filters using faceted search capabilitie...
Github user iraghumitra commented on the issue: https://github.com/apache/metron/pull/710 @nickwallen we can make the fields configurable too. ---
[GitHub] metron issue #710: Metron-1083: Add filters using faceted search capabilitie...
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/710 +1 That works for me. This is a good first pass. Taking care of METRON-1250 would totally side step a probably contentious (and unwinnable) debate as to what the most important fields are. :) ---
[GitHub] metron issue #710: Metron-1083: Add filters using faceted search capabilitie...
Github user james-sirota commented on the issue: https://github.com/apache/metron/pull/710 Ok, I opened https://issues.apache.org/jira/browse/METRON-1250 as a follow on jira for this ---
[GitHub] metron issue #710: Metron-1083: Add filters using faceted search capabilitie...
Github user iraghumitra commented on the issue: https://github.com/apache/metron/pull/710 @nickwallen The idea is to have the Facets and Groups in sync with the table column names at the same time avoid the ones that might be too discrete. I will have a discussion around what's the best way to find out the discrete fields, I don't want a blacklist of fields that would be hardcoded again. Can you raise a jira on my name I would pick it as a follow-up if you feel it can be picked in a follow-up PR? ---
[GitHub] metron issue #768: Metron 1123: Add group by option using faceted search cap...
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/768 @iraghumitra In case you missed it, there are some new merge conflicts based on one of your other PRs being merged into master today. ---
[GitHub] metron issue #710: Metron-1083: Add filters using faceted search capabilitie...
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/710 Everything else is looking really good though, @iraghumitra . * The way that the UI it is laid out is obvious and intuitive. * I manually tested the faceted functionality and all the counts and fields sync'd as I would expect. * It looks like you have solid e2e tests and I was able to run them successfully. ---
[GitHub] metron pull request #768: Metron 1123: Add group by option using faceted sea...
Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/768#discussion_r144644294 --- Diff: metron-interface/metron-alerts/src/app/alerts/alerts-list/tree-view/tree-view.component.ts --- @@ -0,0 +1,319 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { Component, Input, OnChanges, SimpleChanges } from '@angular/core'; +import {Router} from '@angular/router'; +import {Subscription} from 'rxjs/Rx'; + +import {TableViewComponent} from '../table-view/table-view.component'; +import {SearchResponse} from '../../../model/search-response'; +import {SearchService} from '../../../service/search.service'; +import {TreeGroupData, TreeAlertsSubscription} from './tree-group-data'; +import {GroupResponse} from '../../../model/group-response'; +import {GroupResult} from '../../../model/group-result'; +import {Group} from '../../../model/group'; +import {SortField} from '../../../model/sort-field'; +import {Sort} from '../../../utils/enums'; +import {MetronDialogBox, DialogType} from '../../../shared/metron-dialog-box'; +import {ElasticsearchUtils} from '../../../utils/elasticsearch-utils'; +import {SearchRequest} from '../../../model/search-request'; + +@Component({ + selector: 'app-tree-view', + templateUrl: './tree-view.component.html', + styleUrls: ['./tree-view.component.scss'] +}) + +export class TreeViewComponent extends TableViewComponent implements OnChanges { + + groupByFields: string[] = []; + topGroups: TreeGroupData[] = []; + groupResponse: GroupResponse = new GroupResponse(); + treeGroupSubscriptionMap: {[key: string]: TreeAlertsSubscription } = {}; + + constructor(router: Router, + searchService: SearchService, + metronDialogBox: MetronDialogBox) { +super(router, searchService, metronDialogBox); + } + + collapseGroup(groupArray: TreeGroupData[], level: number, index: number) { +for (let i = index + 1; i < groupArray.length; i++) { + if (groupArray[i].level > (level)) { +groupArray[i].show = false; +groupArray[i].expand = false; + } else { +break; + } +} + } + + createQuery(selectedGroup: TreeGroupData) { +let searchQuery = this.queryBuilder.generateSelect(); +let groupQery = Object.keys(selectedGroup.groupQueryMap).map(key => { + return key.replace(/:/g, '\\:') + + ':' + + String(selectedGroup.groupQueryMap[key]) + .replace(/[\*\+\-=~><\"\?^\${}\(\)\:\!\/[\]\\\s]/g, '\\$&') // replace single special characters + .replace(/\|\|/g, '\\||') // replace || + .replace(/\&\&/g, '\\&&'); // replace && +}).join(' AND '); + +groupQery += searchQuery === '*' ? '' : (' AND ' + searchQuery); +return groupQery; + } + + expandGroup(groupArray: TreeGroupData[], level: number, index: number) { +for (let i = index + 1; i < groupArray.length; i++) { + if (groupArray[i].level === (level + 1)) { +groupArray[i].show = true; + } else { +break; + } +} + } + + getAlerts(selectedGroup: TreeGroupData): Subscription { +let searchRequest = new SearchRequest(); +searchRequest.query = this.createQuery(selectedGroup); +searchRequest.from = selectedGroup.pagingData.from; +searchRequest.size = selectedGroup.pagingData.size; +searchRequest.sort = selectedGroup.sortField ? [selectedGroup.sortField] : []; + +return this.searchGroup(selectedGroup, searchRequest); --- End diff -- Yes, that makes sense to me. @merrimanr ---
[GitHub] metron issue #710: Metron-1083: Add filters using faceted search capabilitie...
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/710 > A new facet pane is added to GUI that groups data based on the following 'hard coded' fields ('source:type', 'ip_src_addr', 'ip_dst_addr', 'host', 'enrichments:geo:ip_dst_addr:country') I am assuming that we will "un-hard code" the list of fields included in the facet panel in a future PR. But even as a first pass, I don't think 'host' should be included. This is not a common field. It is only present in a subset of Bro records. Most records do not have this field. ---
[GitHub] metron pull request #768: Metron 1123: Add group by option using faceted sea...
Github user merrimanr commented on a diff in the pull request: https://github.com/apache/metron/pull/768#discussion_r144639257 --- Diff: metron-interface/metron-alerts/src/app/alerts/alerts-list/tree-view/tree-view.component.ts --- @@ -0,0 +1,319 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { Component, Input, OnChanges, SimpleChanges } from '@angular/core'; +import {Router} from '@angular/router'; +import {Subscription} from 'rxjs/Rx'; + +import {TableViewComponent} from '../table-view/table-view.component'; +import {SearchResponse} from '../../../model/search-response'; +import {SearchService} from '../../../service/search.service'; +import {TreeGroupData, TreeAlertsSubscription} from './tree-group-data'; +import {GroupResponse} from '../../../model/group-response'; +import {GroupResult} from '../../../model/group-result'; +import {Group} from '../../../model/group'; +import {SortField} from '../../../model/sort-field'; +import {Sort} from '../../../utils/enums'; +import {MetronDialogBox, DialogType} from '../../../shared/metron-dialog-box'; +import {ElasticsearchUtils} from '../../../utils/elasticsearch-utils'; +import {SearchRequest} from '../../../model/search-request'; + +@Component({ + selector: 'app-tree-view', + templateUrl: './tree-view.component.html', + styleUrls: ['./tree-view.component.scss'] +}) + +export class TreeViewComponent extends TableViewComponent implements OnChanges { + + groupByFields: string[] = []; + topGroups: TreeGroupData[] = []; + groupResponse: GroupResponse = new GroupResponse(); + treeGroupSubscriptionMap: {[key: string]: TreeAlertsSubscription } = {}; + + constructor(router: Router, + searchService: SearchService, + metronDialogBox: MetronDialogBox) { +super(router, searchService, metronDialogBox); + } + + collapseGroup(groupArray: TreeGroupData[], level: number, index: number) { +for (let i = index + 1; i < groupArray.length; i++) { + if (groupArray[i].level > (level)) { +groupArray[i].show = false; +groupArray[i].expand = false; + } else { +break; + } +} + } + + createQuery(selectedGroup: TreeGroupData) { +let searchQuery = this.queryBuilder.generateSelect(); +let groupQery = Object.keys(selectedGroup.groupQueryMap).map(key => { + return key.replace(/:/g, '\\:') + + ':' + + String(selectedGroup.groupQueryMap[key]) + .replace(/[\*\+\-=~><\"\?^\${}\(\)\:\!\/[\]\\\s]/g, '\\$&') // replace single special characters + .replace(/\|\|/g, '\\||') // replace || + .replace(/\&\&/g, '\\&&'); // replace && +}).join(' AND '); + +groupQery += searchQuery === '*' ? '' : (' AND ' + searchQuery); +return groupQery; + } + + expandGroup(groupArray: TreeGroupData[], level: number, index: number) { +for (let i = index + 1; i < groupArray.length; i++) { + if (groupArray[i].level === (level + 1)) { +groupArray[i].show = true; + } else { +break; + } +} + } + + getAlerts(selectedGroup: TreeGroupData): Subscription { +let searchRequest = new SearchRequest(); +searchRequest.query = this.createQuery(selectedGroup); +searchRequest.from = selectedGroup.pagingData.from; +searchRequest.size = selectedGroup.pagingData.size; +searchRequest.sort = selectedGroup.sortField ? [selectedGroup.sortField] : []; + +return this.searchGroup(selectedGroup, searchRequest); --- End diff -- I believe this issue is similar to https://github.com/apache/metron/pull/768#issuecomment-335281376 and should be resolved in the same follow on PR. Do you agree @nickwallen? ---
Re: [DISCUSS] Build broken due to transitive dependencies
Does that mean CentOS 6 won't work anymore? On 2017-10-13 11:12, Ryan Merriman wrote: We recently ran into this and the cause was an old C++ compiler version. It wants a compiler that has support for C++11: https://gcc.gnu.org/projects/cxx-status.html#cxx11. On Fri, Oct 13, 2017 at 1:00 PM, Laurens Vets wrote: ... [INFO] --- frontend-maven-plugin:1.3:npm (ng build) @ metron-config --- [DEBUG] Configuring mojo com.github.eirslett:frontend-maven-plugin:1.3:npm from plugin realm ClassRealm[plugin>com.github.e irslett:frontend-maven-plugin:1.3, parent: sun.misc.Launcher$AppClassLoad er@70dea4e] [DEBUG] Configuring mojo 'com.github.eirslett:frontend-maven-plugin:1.3:npm' with basic configurator --> [DEBUG] (f) arguments = run build [DEBUG] (f) npmInheritsProxyConfigFromMaven = false [DEBUG] (f) project = MavenProject: org.apache.metron:metron-config:0.4.1 @ /root/metron/metron-interface/metron-config/pom.xml [DEBUG] (f) repositorySystemSession = org.eclipse.aether.DefaultRepo sitorySystemSession@e883a51 [DEBUG] (f) session = org.apache.maven.execution.MavenSession@2aaefbd [DEBUG] (f) skip = false [DEBUG] (f) skipTests = true [DEBUG] (f) workingDirectory = /root/metron/metron-interface/ metron-config [DEBUG] (f) execution = com.github.eirslett:frontend-maven-plugin:1.3:npm {execution: ng build} [DEBUG] -- end configuration -- [INFO] npm not inheriting proxy config from Maven [INFO] Running 'npm run build' in /root/metron/metron-interface/ metron-config [INFO] [INFO] > metron-management-ui@0.4.1 build /root/metron/metron-interface/ metron-config [INFO] > ./node_modules/angular-cli/bin/ng build -prod [INFO] [INFO] Cannot find module 'tough-cookie' [INFO] Error: Cannot find module 'tough-cookie' [INFO] at Function.Module._resolveFilename (module.js:440:15) [INFO] at Function.Module._load (module.js:388:25) [INFO] at Module.require (module.js:468:17) [INFO] at require (internal/module.js:20:19) [INFO] at Object. (/root/metron/metron-interface /metron-config/node_modules/request/lib/cookies.js:3:13) [INFO] at Module._compile (module.js:541:32) [INFO] at Object.Module._extensions..js (module.js:550:10) [INFO] at Module.load (module.js:458:32) [INFO] at tryModuleLoad (module.js:417:12) [INFO] at Function.Module._load (module.js:409:3) [INFO] at Module.require (module.js:468:17) [INFO] at require (internal/module.js:20:19) [INFO] at Object. (/root/metron/metron-interface /metron-config/node_modules/request/index.js:18:15) [INFO] at Module._compile (module.js:541:32) [INFO] at Object.Module._extensions..js (module.js:550:10) [INFO] at Module.load (module.js:458:32) [INFO] at tryModuleLoad (module.js:417:12) [INFO] at Function.Module._load (module.js:409:3) [INFO] at Module.require (module.js:468:17) [INFO] at require (internal/module.js:20:19) [INFO] at Leek._enqueue (/root/metron/metron-interface /metron-config/node_modules/leek/lib/leek.js:60:30) [INFO] at Leek.track (/root/metron/metron-interface /metron-config/node_modules/leek/lib/leek.js:87:15) [INFO] at Class.Command.validateAndRun (/root/metron/metron-interface /metron-config/node_modules/angular-cli/lib/models/command.js:119:18) [INFO] at /root/metron/metron-interface/metron-config/node_modules/ang ular-cli/lib/cli/cli.js:86:22 [INFO] at tryCatch (/root/metron/metron-interface /metron-config/node_modules/rsvp/dist/lib/rsvp/-internal.js:198:12) [INFO] at invokeCallback (/root/metron/metron-interface /metron-config/node_modules/rsvp/dist/lib/rsvp/-internal.js:211:13) [INFO] at /root/metron/metron-interface/metron-config/node_modules/rsv p/dist/lib/rsvp/then.js:26:14 [INFO] at flush (/root/metron/metron-interface /metron-config/node_modules/rsvp/dist/lib/rsvp/asap.js:80:5) [INFO] at _combinedTickCallback (internal/process/next_tick.js:67:7) [INFO] at process._tickCallback (internal/process/next_tick.js:98:9) [ERROR] [ERROR] npm ERR! Linux 2.6.32-696.13.2.el6.x86_64 [ERROR] npm ERR! argv "/root/metron/metron-interface/metron-config/node/node" "/root/metron/metron-interface/metron-config/node/node_modules/npm/bin/npm-cli.js" "run" "build" [ERROR] npm ERR! node v6.2.0 [ERROR] npm ERR! npm v3.8.9 [ERROR] npm ERR! code ELIFECYCLE [ERROR] npm ERR! metron-management-ui@0.4.1 build: `./node_modules/angular-cli/bin/ng build -prod` [ERROR] npm ERR! Exit status 1 [ERROR] npm ERR! [ERROR] npm ERR! Failed at the metron-management-ui@0.4.1 build script './node_modules/angular-cli/bin/ng build -prod'. [ERROR] npm ERR! Make sure you have the latest version of node.js and npm installed. [ERROR] npm ERR! If you do, this is most likely a problem with the metron-management-ui package, [ERROR] npm ERR! not with npm itself. [ERROR] npm ERR! Tell the author that this fails on your system: [ERROR] npm ERR! ./node_modules/angular-cli/bin/ng build -prod [ERROR] npm ERR! You can get information on how to open an i
[GitHub] metron issue #710: Metron-1083: Add filters using faceted search capabilitie...
Github user iraghumitra commented on the issue: https://github.com/apache/metron/pull/710 @nickwallen The PR was raised before I go to know about the concept of squash commits mind me for my ignorance. I had to make few modifications since a PR that was raised after this got in to master before. I tested the PR again looks good feel free to review. ---
[GitHub] metron pull request #768: Metron 1123: Add group by option using faceted sea...
Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/768#discussion_r144633129 --- Diff: metron-interface/metron-alerts/src/app/alerts/alerts-list/tree-view/tree-view.component.ts --- @@ -0,0 +1,319 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { Component, Input, OnChanges, SimpleChanges } from '@angular/core'; +import {Router} from '@angular/router'; +import {Subscription} from 'rxjs/Rx'; + +import {TableViewComponent} from '../table-view/table-view.component'; +import {SearchResponse} from '../../../model/search-response'; +import {SearchService} from '../../../service/search.service'; +import {TreeGroupData, TreeAlertsSubscription} from './tree-group-data'; +import {GroupResponse} from '../../../model/group-response'; +import {GroupResult} from '../../../model/group-result'; +import {Group} from '../../../model/group'; +import {SortField} from '../../../model/sort-field'; +import {Sort} from '../../../utils/enums'; +import {MetronDialogBox, DialogType} from '../../../shared/metron-dialog-box'; +import {ElasticsearchUtils} from '../../../utils/elasticsearch-utils'; +import {SearchRequest} from '../../../model/search-request'; + +@Component({ + selector: 'app-tree-view', + templateUrl: './tree-view.component.html', + styleUrls: ['./tree-view.component.scss'] +}) + +export class TreeViewComponent extends TableViewComponent implements OnChanges { + + groupByFields: string[] = []; + topGroups: TreeGroupData[] = []; + groupResponse: GroupResponse = new GroupResponse(); + treeGroupSubscriptionMap: {[key: string]: TreeAlertsSubscription } = {}; + + constructor(router: Router, + searchService: SearchService, + metronDialogBox: MetronDialogBox) { +super(router, searchService, metronDialogBox); + } + + collapseGroup(groupArray: TreeGroupData[], level: number, index: number) { +for (let i = index + 1; i < groupArray.length; i++) { + if (groupArray[i].level > (level)) { +groupArray[i].show = false; +groupArray[i].expand = false; + } else { +break; + } +} + } + + createQuery(selectedGroup: TreeGroupData) { +let searchQuery = this.queryBuilder.generateSelect(); +let groupQery = Object.keys(selectedGroup.groupQueryMap).map(key => { + return key.replace(/:/g, '\\:') + + ':' + + String(selectedGroup.groupQueryMap[key]) + .replace(/[\*\+\-=~><\"\?^\${}\(\)\:\!\/[\]\\\s]/g, '\\$&') // replace single special characters + .replace(/\|\|/g, '\\||') // replace || + .replace(/\&\&/g, '\\&&'); // replace && +}).join(' AND '); + +groupQery += searchQuery === '*' ? '' : (' AND ' + searchQuery); +return groupQery; + } + + expandGroup(groupArray: TreeGroupData[], level: number, index: number) { +for (let i = index + 1; i < groupArray.length; i++) { + if (groupArray[i].level === (level + 1)) { +groupArray[i].show = true; + } else { +break; + } +} + } + + getAlerts(selectedGroup: TreeGroupData): Subscription { +let searchRequest = new SearchRequest(); +searchRequest.query = this.createQuery(selectedGroup); +searchRequest.from = selectedGroup.pagingData.from; +searchRequest.size = selectedGroup.pagingData.size; +searchRequest.sort = selectedGroup.sortField ? [selectedGroup.sortField] : []; + +return this.searchGroup(selectedGroup, searchRequest); --- End diff -- @iraghumitra That doesn't explain this one, right? https://github.com/apache/metron/pull/768#discussion_r144387064 ---
Re: [DISCUSS] Build broken due to transitive dependencies
We recently ran into this and the cause was an old C++ compiler version. It wants a compiler that has support for C++11: https://gcc.gnu.org/projects/cxx-status.html#cxx11. On Fri, Oct 13, 2017 at 1:00 PM, Laurens Vets wrote: > ... > [INFO] --- frontend-maven-plugin:1.3:npm (ng build) @ metron-config --- > [DEBUG] Configuring mojo com.github.eirslett:frontend-maven-plugin:1.3:npm > from plugin realm ClassRealm[plugin>com.github.e > irslett:frontend-maven-plugin:1.3, parent: sun.misc.Launcher$AppClassLoad > er@70dea4e] > [DEBUG] Configuring mojo 'com.github.eirslett:frontend-maven-plugin:1.3:npm' > with basic configurator --> > [DEBUG] (f) arguments = run build > [DEBUG] (f) npmInheritsProxyConfigFromMaven = false > [DEBUG] (f) project = MavenProject: org.apache.metron:metron-config:0.4.1 > @ /root/metron/metron-interface/metron-config/pom.xml > [DEBUG] (f) repositorySystemSession = org.eclipse.aether.DefaultRepo > sitorySystemSession@e883a51 > [DEBUG] (f) session = org.apache.maven.execution.MavenSession@2aaefbd > [DEBUG] (f) skip = false > [DEBUG] (f) skipTests = true > [DEBUG] (f) workingDirectory = /root/metron/metron-interface/ > metron-config > [DEBUG] (f) execution = com.github.eirslett:frontend-maven-plugin:1.3:npm > {execution: ng build} > [DEBUG] -- end configuration -- > [INFO] npm not inheriting proxy config from Maven > [INFO] Running 'npm run build' in /root/metron/metron-interface/ > metron-config > [INFO] > [INFO] > metron-management-ui@0.4.1 build /root/metron/metron-interface/ > metron-config > [INFO] > ./node_modules/angular-cli/bin/ng build -prod > [INFO] > [INFO] Cannot find module 'tough-cookie' > [INFO] Error: Cannot find module 'tough-cookie' > [INFO] at Function.Module._resolveFilename (module.js:440:15) > [INFO] at Function.Module._load (module.js:388:25) > [INFO] at Module.require (module.js:468:17) > [INFO] at require (internal/module.js:20:19) > [INFO] at Object. (/root/metron/metron-interface > /metron-config/node_modules/request/lib/cookies.js:3:13) > [INFO] at Module._compile (module.js:541:32) > [INFO] at Object.Module._extensions..js (module.js:550:10) > [INFO] at Module.load (module.js:458:32) > [INFO] at tryModuleLoad (module.js:417:12) > [INFO] at Function.Module._load (module.js:409:3) > [INFO] at Module.require (module.js:468:17) > [INFO] at require (internal/module.js:20:19) > [INFO] at Object. (/root/metron/metron-interface > /metron-config/node_modules/request/index.js:18:15) > [INFO] at Module._compile (module.js:541:32) > [INFO] at Object.Module._extensions..js (module.js:550:10) > [INFO] at Module.load (module.js:458:32) > [INFO] at tryModuleLoad (module.js:417:12) > [INFO] at Function.Module._load (module.js:409:3) > [INFO] at Module.require (module.js:468:17) > [INFO] at require (internal/module.js:20:19) > [INFO] at Leek._enqueue (/root/metron/metron-interface > /metron-config/node_modules/leek/lib/leek.js:60:30) > [INFO] at Leek.track (/root/metron/metron-interface > /metron-config/node_modules/leek/lib/leek.js:87:15) > [INFO] at Class.Command.validateAndRun (/root/metron/metron-interface > /metron-config/node_modules/angular-cli/lib/models/command.js:119:18) > [INFO] at /root/metron/metron-interface/metron-config/node_modules/ang > ular-cli/lib/cli/cli.js:86:22 > [INFO] at tryCatch (/root/metron/metron-interface > /metron-config/node_modules/rsvp/dist/lib/rsvp/-internal.js:198:12) > [INFO] at invokeCallback (/root/metron/metron-interface > /metron-config/node_modules/rsvp/dist/lib/rsvp/-internal.js:211:13) > [INFO] at /root/metron/metron-interface/metron-config/node_modules/rsv > p/dist/lib/rsvp/then.js:26:14 > [INFO] at flush (/root/metron/metron-interface > /metron-config/node_modules/rsvp/dist/lib/rsvp/asap.js:80:5) > [INFO] at _combinedTickCallback (internal/process/next_tick.js:67:7) > [INFO] at process._tickCallback (internal/process/next_tick.js:98:9) > [ERROR] > [ERROR] npm ERR! Linux 2.6.32-696.13.2.el6.x86_64 > [ERROR] npm ERR! argv "/root/metron/metron-interface/metron-config/node/node" > "/root/metron/metron-interface/metron-config/node/node_modules/npm/bin/npm-cli.js" > "run" "build" > [ERROR] npm ERR! node v6.2.0 > [ERROR] npm ERR! npm v3.8.9 > [ERROR] npm ERR! code ELIFECYCLE > [ERROR] npm ERR! metron-management-ui@0.4.1 build: > `./node_modules/angular-cli/bin/ng build -prod` > [ERROR] npm ERR! Exit status 1 > [ERROR] npm ERR! > [ERROR] npm ERR! Failed at the metron-management-ui@0.4.1 build script > './node_modules/angular-cli/bin/ng build -prod'. > [ERROR] npm ERR! Make sure you have the latest version of node.js and npm > installed. > [ERROR] npm ERR! If you do, this is most likely a problem with the > metron-management-ui package, > [ERROR] npm ERR! not with npm itself. > [ERROR] npm ERR! Tell the author that this fails on your system: > [ERROR] npm ERR! ./node_modules/angular-cli/bin/ng build -
Re: [DISCUSS] Build broken due to transitive dependencies
[INFO] Cannot find module 'tough-cookie' - It's as if the build has become sentient and now mocks us On Fri, Oct 13, 2017 at 12:00 PM, Laurens Vets wrote: > ... > [INFO] --- frontend-maven-plugin:1.3:npm (ng build) @ metron-config --- > [DEBUG] Configuring mojo com.github.eirslett:frontend-maven-plugin:1.3:npm > from plugin realm ClassRealm[plugin>com.github.e > irslett:frontend-maven-plugin:1.3, parent: sun.misc.Launcher$AppClassLoad > er@70dea4e] > [DEBUG] Configuring mojo 'com.github.eirslett:frontend-maven-plugin:1.3:npm' > with basic configurator --> > [DEBUG] (f) arguments = run build > [DEBUG] (f) npmInheritsProxyConfigFromMaven = false > [DEBUG] (f) project = MavenProject: org.apache.metron:metron-config:0.4.1 > @ /root/metron/metron-interface/metron-config/pom.xml > [DEBUG] (f) repositorySystemSession = org.eclipse.aether.DefaultRepo > sitorySystemSession@e883a51 > [DEBUG] (f) session = org.apache.maven.execution.MavenSession@2aaefbd > [DEBUG] (f) skip = false > [DEBUG] (f) skipTests = true > [DEBUG] (f) workingDirectory = /root/metron/metron-interface/ > metron-config > [DEBUG] (f) execution = com.github.eirslett:frontend-maven-plugin:1.3:npm > {execution: ng build} > [DEBUG] -- end configuration -- > [INFO] npm not inheriting proxy config from Maven > [INFO] Running 'npm run build' in /root/metron/metron-interface/ > metron-config > [INFO] > [INFO] > metron-management-ui@0.4.1 build /root/metron/metron-interface/ > metron-config > [INFO] > ./node_modules/angular-cli/bin/ng build -prod > [INFO] > [INFO] Cannot find module 'tough-cookie' > [INFO] Error: Cannot find module 'tough-cookie' > [INFO] at Function.Module._resolveFilename (module.js:440:15) > [INFO] at Function.Module._load (module.js:388:25) > [INFO] at Module.require (module.js:468:17) > [INFO] at require (internal/module.js:20:19) > [INFO] at Object. (/root/metron/metron-interface > /metron-config/node_modules/request/lib/cookies.js:3:13) > [INFO] at Module._compile (module.js:541:32) > [INFO] at Object.Module._extensions..js (module.js:550:10) > [INFO] at Module.load (module.js:458:32) > [INFO] at tryModuleLoad (module.js:417:12) > [INFO] at Function.Module._load (module.js:409:3) > [INFO] at Module.require (module.js:468:17) > [INFO] at require (internal/module.js:20:19) > [INFO] at Object. (/root/metron/metron-interface > /metron-config/node_modules/request/index.js:18:15) > [INFO] at Module._compile (module.js:541:32) > [INFO] at Object.Module._extensions..js (module.js:550:10) > [INFO] at Module.load (module.js:458:32) > [INFO] at tryModuleLoad (module.js:417:12) > [INFO] at Function.Module._load (module.js:409:3) > [INFO] at Module.require (module.js:468:17) > [INFO] at require (internal/module.js:20:19) > [INFO] at Leek._enqueue (/root/metron/metron-interface > /metron-config/node_modules/leek/lib/leek.js:60:30) > [INFO] at Leek.track (/root/metron/metron-interface > /metron-config/node_modules/leek/lib/leek.js:87:15) > [INFO] at Class.Command.validateAndRun (/root/metron/metron-interface > /metron-config/node_modules/angular-cli/lib/models/command.js:119:18) > [INFO] at /root/metron/metron-interface/metron-config/node_modules/ang > ular-cli/lib/cli/cli.js:86:22 > [INFO] at tryCatch (/root/metron/metron-interface > /metron-config/node_modules/rsvp/dist/lib/rsvp/-internal.js:198:12) > [INFO] at invokeCallback (/root/metron/metron-interface > /metron-config/node_modules/rsvp/dist/lib/rsvp/-internal.js:211:13) > [INFO] at /root/metron/metron-interface/metron-config/node_modules/rsv > p/dist/lib/rsvp/then.js:26:14 > [INFO] at flush (/root/metron/metron-interface > /metron-config/node_modules/rsvp/dist/lib/rsvp/asap.js:80:5) > [INFO] at _combinedTickCallback (internal/process/next_tick.js:67:7) > [INFO] at process._tickCallback (internal/process/next_tick.js:98:9) > [ERROR] > [ERROR] npm ERR! Linux 2.6.32-696.13.2.el6.x86_64 > [ERROR] npm ERR! argv "/root/metron/metron-interface/metron-config/node/node" > "/root/metron/metron-interface/metron-config/node/node_modules/npm/bin/npm-cli.js" > "run" "build" > [ERROR] npm ERR! node v6.2.0 > [ERROR] npm ERR! npm v3.8.9 > [ERROR] npm ERR! code ELIFECYCLE > [ERROR] npm ERR! metron-management-ui@0.4.1 build: > `./node_modules/angular-cli/bin/ng build -prod` > [ERROR] npm ERR! Exit status 1 > [ERROR] npm ERR! > [ERROR] npm ERR! Failed at the metron-management-ui@0.4.1 build script > './node_modules/angular-cli/bin/ng build -prod'. > [ERROR] npm ERR! Make sure you have the latest version of node.js and npm > installed. > [ERROR] npm ERR! If you do, this is most likely a problem with the > metron-management-ui package, > [ERROR] npm ERR! not with npm itself. > [ERROR] npm ERR! Tell the author that this fails on your system: > [ERROR] npm ERR! ./node_modules/angular-cli/bin/ng build -prod > [ERROR] npm ERR! You can get information on how to open an issue
[GitHub] metron pull request #787: METRON-1232: Alert status changes are not reflecte...
Github user asfgit closed the pull request at: https://github.com/apache/metron/pull/787 ---
Re: [DISCUSS] Build broken due to transitive dependencies
... [INFO] --- frontend-maven-plugin:1.3:npm (ng build) @ metron-config --- [DEBUG] Configuring mojo com.github.eirslett:frontend-maven-plugin:1.3:npm from plugin realm ClassRealm[plugin>com.github.eirslett:frontend-maven-plugin:1.3, parent: sun.misc.Launcher$AppClassLoader@70dea4e] [DEBUG] Configuring mojo 'com.github.eirslett:frontend-maven-plugin:1.3:npm' with basic configurator --> [DEBUG] (f) arguments = run build [DEBUG] (f) npmInheritsProxyConfigFromMaven = false [DEBUG] (f) project = MavenProject: org.apache.metron:metron-config:0.4.1 @ /root/metron/metron-interface/metron-config/pom.xml [DEBUG] (f) repositorySystemSession = org.eclipse.aether.DefaultRepositorySystemSession@e883a51 [DEBUG] (f) session = org.apache.maven.execution.MavenSession@2aaefbd [DEBUG] (f) skip = false [DEBUG] (f) skipTests = true [DEBUG] (f) workingDirectory = /root/metron/metron-interface/metron-config [DEBUG] (f) execution = com.github.eirslett:frontend-maven-plugin:1.3:npm {execution: ng build} [DEBUG] -- end configuration -- [INFO] npm not inheriting proxy config from Maven [INFO] Running 'npm run build' in /root/metron/metron-interface/metron-config [INFO] [INFO] > metron-management-ui@0.4.1 build /root/metron/metron-interface/metron-config [INFO] > ./node_modules/angular-cli/bin/ng build -prod [INFO] [INFO] Cannot find module 'tough-cookie' [INFO] Error: Cannot find module 'tough-cookie' [INFO] at Function.Module._resolveFilename (module.js:440:15) [INFO] at Function.Module._load (module.js:388:25) [INFO] at Module.require (module.js:468:17) [INFO] at require (internal/module.js:20:19) [INFO] at Object. (/root/metron/metron-interface/metron-config/node_modules/request/lib/cookies.js:3:13) [INFO] at Module._compile (module.js:541:32) [INFO] at Object.Module._extensions..js (module.js:550:10) [INFO] at Module.load (module.js:458:32) [INFO] at tryModuleLoad (module.js:417:12) [INFO] at Function.Module._load (module.js:409:3) [INFO] at Module.require (module.js:468:17) [INFO] at require (internal/module.js:20:19) [INFO] at Object. (/root/metron/metron-interface/metron-config/node_modules/request/index.js:18:15) [INFO] at Module._compile (module.js:541:32) [INFO] at Object.Module._extensions..js (module.js:550:10) [INFO] at Module.load (module.js:458:32) [INFO] at tryModuleLoad (module.js:417:12) [INFO] at Function.Module._load (module.js:409:3) [INFO] at Module.require (module.js:468:17) [INFO] at require (internal/module.js:20:19) [INFO] at Leek._enqueue (/root/metron/metron-interface/metron-config/node_modules/leek/lib/leek.js:60:30) [INFO] at Leek.track (/root/metron/metron-interface/metron-config/node_modules/leek/lib/leek.js:87:15) [INFO] at Class.Command.validateAndRun (/root/metron/metron-interface/metron-config/node_modules/angular-cli/lib/models/command.js:119:18) [INFO] at /root/metron/metron-interface/metron-config/node_modules/angular-cli/lib/cli/cli.js:86:22 [INFO] at tryCatch (/root/metron/metron-interface/metron-config/node_modules/rsvp/dist/lib/rsvp/-internal.js:198:12) [INFO] at invokeCallback (/root/metron/metron-interface/metron-config/node_modules/rsvp/dist/lib/rsvp/-internal.js:211:13) [INFO] at /root/metron/metron-interface/metron-config/node_modules/rsvp/dist/lib/rsvp/then.js:26:14 [INFO] at flush (/root/metron/metron-interface/metron-config/node_modules/rsvp/dist/lib/rsvp/asap.js:80:5) [INFO] at _combinedTickCallback (internal/process/next_tick.js:67:7) [INFO] at process._tickCallback (internal/process/next_tick.js:98:9) [ERROR] [ERROR] npm ERR! Linux 2.6.32-696.13.2.el6.x86_64 [ERROR] npm ERR! argv "/root/metron/metron-interface/metron-config/node/node" "/root/metron/metron-interface/metron-config/node/node_modules/npm/bin/npm-cli.js" "run" "build" [ERROR] npm ERR! node v6.2.0 [ERROR] npm ERR! npm v3.8.9 [ERROR] npm ERR! code ELIFECYCLE [ERROR] npm ERR! metron-management-ui@0.4.1 build: `./node_modules/angular-cli/bin/ng build -prod` [ERROR] npm ERR! Exit status 1 [ERROR] npm ERR! [ERROR] npm ERR! Failed at the metron-management-ui@0.4.1 build script './node_modules/angular-cli/bin/ng build -prod'. [ERROR] npm ERR! Make sure you have the latest version of node.js and npm installed. [ERROR] npm ERR! If you do, this is most likely a problem with the metron-management-ui package, [ERROR] npm ERR! not with npm itself. [ERROR] npm ERR! Tell the author that this fails on your system: [ERROR] npm ERR! ./node_modules/angular-cli/bin/ng build -prod [ERROR] npm ERR! You can get information on how to open an issue for this project with: [ERROR] npm ERR! npm bugs metron-management-ui [ERROR] npm ERR! Or if that isn't available, you can get their info via: [ERROR] npm ERR! npm owner ls metron-management-ui [ERROR] npm ERR! There is likely additional logging output above. [ERROR] [ERROR] npm ERR! Please include the follow
[GitHub] metron issue #710: Metron-1083: Add filters using faceted search capabilitie...
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/710 Hi @iraghumitra - The commit history looks a little odd. There are commits going back a few months and seem unrelated to this PR. Can you double check that the code that you have submitted is what you expect? ---
[GitHub] metron pull request #768: Metron 1123: Add group by option using faceted sea...
Github user iraghumitra commented on a diff in the pull request: https://github.com/apache/metron/pull/768#discussion_r144592599 --- Diff: metron-interface/metron-alerts/src/app/alerts/alerts-list/tree-view/tree-view.component.ts --- @@ -0,0 +1,319 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { Component, Input, OnChanges, SimpleChanges } from '@angular/core'; +import {Router} from '@angular/router'; +import {Subscription} from 'rxjs/Rx'; + +import {TableViewComponent} from '../table-view/table-view.component'; +import {SearchResponse} from '../../../model/search-response'; +import {SearchService} from '../../../service/search.service'; +import {TreeGroupData, TreeAlertsSubscription} from './tree-group-data'; +import {GroupResponse} from '../../../model/group-response'; +import {GroupResult} from '../../../model/group-result'; +import {Group} from '../../../model/group'; +import {SortField} from '../../../model/sort-field'; +import {Sort} from '../../../utils/enums'; +import {MetronDialogBox, DialogType} from '../../../shared/metron-dialog-box'; +import {ElasticsearchUtils} from '../../../utils/elasticsearch-utils'; +import {SearchRequest} from '../../../model/search-request'; + +@Component({ + selector: 'app-tree-view', + templateUrl: './tree-view.component.html', + styleUrls: ['./tree-view.component.scss'] +}) + +export class TreeViewComponent extends TableViewComponent implements OnChanges { + + groupByFields: string[] = []; + topGroups: TreeGroupData[] = []; + groupResponse: GroupResponse = new GroupResponse(); + treeGroupSubscriptionMap: {[key: string]: TreeAlertsSubscription } = {}; + + constructor(router: Router, + searchService: SearchService, + metronDialogBox: MetronDialogBox) { +super(router, searchService, metronDialogBox); + } + + collapseGroup(groupArray: TreeGroupData[], level: number, index: number) { +for (let i = index + 1; i < groupArray.length; i++) { + if (groupArray[i].level > (level)) { +groupArray[i].show = false; +groupArray[i].expand = false; + } else { +break; + } +} + } + + createQuery(selectedGroup: TreeGroupData) { +let searchQuery = this.queryBuilder.generateSelect(); +let groupQery = Object.keys(selectedGroup.groupQueryMap).map(key => { + return key.replace(/:/g, '\\:') + + ':' + + String(selectedGroup.groupQueryMap[key]) + .replace(/[\*\+\-=~><\"\?^\${}\(\)\:\!\/[\]\\\s]/g, '\\$&') // replace single special characters + .replace(/\|\|/g, '\\||') // replace || + .replace(/\&\&/g, '\\&&'); // replace && +}).join(' AND '); + +groupQery += searchQuery === '*' ? '' : (' AND ' + searchQuery); +return groupQery; + } + + expandGroup(groupArray: TreeGroupData[], level: number, index: number) { +for (let i = index + 1; i < groupArray.length; i++) { + if (groupArray[i].level === (level + 1)) { +groupArray[i].show = true; + } else { +break; + } +} + } + + getAlerts(selectedGroup: TreeGroupData): Subscription { +let searchRequest = new SearchRequest(); +searchRequest.query = this.createQuery(selectedGroup); +searchRequest.from = selectedGroup.pagingData.from; +searchRequest.size = selectedGroup.pagingData.size; +searchRequest.sort = selectedGroup.sortField ? [selectedGroup.sortField] : []; + +return this.searchGroup(selectedGroup, searchRequest); --- End diff -- @nickwallen Again the problem is with how we are handling null values in the rest api. The host name for all the alerts is emty so we are not seeing any alerts there. ---
[GitHub] metron pull request #798: METRON-1247: REST search and findOne endpoints ret...
Github user asfgit closed the pull request at: https://github.com/apache/metron/pull/798 ---
[GitHub] metron issue #798: METRON-1247: REST search and findOne endpoints return une...
Github user cestella commented on the issue: https://github.com/apache/metron/pull/798 +1 by inspection, good job! ---