[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-08-10 Thread nickwallen
Github user nickwallen commented on the issue:

https://github.com/apache/metron/pull/620
  
Great.  This is good to go.  Going to merge now.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-08-09 Thread merrimanr
Github user merrimanr commented on the issue:

https://github.com/apache/metron/pull/620
  
+1 nice work @iraghumitra 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-08-09 Thread ottobackwards
Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/620
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-08-09 Thread simonellistonball
Github user simonellistonball commented on the issue:

https://github.com/apache/metron/pull/620
  
+1 I'm good with this. My one niggle will be dealt with by other follow on 
issues.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-08-09 Thread cestella
Github user cestella commented on the issue:

https://github.com/apache/metron/pull/620
  
+1 by inspection, great job @iraghumitra 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-08-08 Thread BlackHatJohnDoe
Github user BlackHatJohnDoe commented on the issue:

https://github.com/apache/metron/pull/620
  
My request was implemented but other than that, I didn't test this 
comprehensively enough to vote on it.  I will hold my peace (and excitement) 
for this PR =)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-08-08 Thread nickwallen
Github user nickwallen commented on the issue:

https://github.com/apache/metron/pull/620
  

+1  This looks great, Raghu!  Nice work.  I look forward to getting this 
large contribution merged in.

Before we merge, let's try to get a nod from everyone else who chimed in on 
the PR.  Speak now or forever hold your peace. ;)
- [ ] @ottobackwards 
- [ ] @simonellistonball 
- [ ] @merrimanr 
- [ ] @BlackHatJohnDoe 
- [ ] @cestella 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-08-08 Thread nickwallen
Github user nickwallen commented on the issue:

https://github.com/apache/metron/pull/620
  
@iraghumitra Thanks for clarifying the licensing issue.  LGTM

@ottobackwards An automated license check for NPM packages would be a nice 
addition.  Probably a good follow-on.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-08-08 Thread iraghumitra
Github user iraghumitra commented on the issue:

https://github.com/apache/metron/pull/620
  
@nickwallen metron-alerts package.json already mentions that it is  Apache 
2.0 licensed. The license-checker tool reports the license of private repos as 
'UNLICENSED' hence the issue. You can confirm this behavior of license-checker 
[here](https://github.com/davglass/license-checker/pull/108).

I changed the license name in package.json file from 'Apache 2.0' to 
'Apache-2.0' to make it consistent with other packages.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-08-07 Thread ottobackwards
Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/620
  
These lic. check commands needs to make their way into the build, like we 
do with the java stuff. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-08-07 Thread nickwallen
Github user nickwallen commented on the issue:

https://github.com/apache/metron/pull/620
  
If we just focus on the production dependencies, which AFAIK is all we need 
to worry about, then I think the licenses are not a problem.

```
$ license-checker --production | grep licenses | sort | uniq -c | sort -n
   1├─ licenses: MIT
   1 │  ├─ licenses: (OFL-1.1 AND MIT)
   1 │  ├─ licenses: BSD*
   1 │  ├─ licenses: UNLICENSED
   3 │  ├─ licenses: Apache-2.0
  13 │  ├─ licenses: MIT
```

The single `UNLICENSED` dependency is actually `metron-alerts` itself.  Can 
you correct that?  

```
├─ metron-alerts@0.4.0
│  ├─ licenses: UNLICENSED
│  ├─ private: true
│  ├─ licenseFile: 
/Users/nallen/tmp/metron-pr620/metron-interface/metron-alerts/README.md
│  └─ noticeFile: 
/Users/nallen/tmp/metron-pr620/metron-interface/metron-alerts/NOTICE
```




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-27 Thread iraghumitra
Github user iraghumitra commented on the issue:

https://github.com/apache/metron/pull/620
  
@nickwallen The e2e tests are working fine i checked them again you need to 
do the below two setps in two consoles.
```
metron-interface/metron-alerts » ./scripts/start-server-for-e2e.sh
metron-interface/metron-alerts » npm run e2e
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-27 Thread iraghumitra
Github user iraghumitra commented on the issue:

https://github.com/apache/metron/pull/620
  
@mraliagha The rest interface for retrieving elastic data is on the way I 
would suggest we should handle it over there. If you have any simpler way to 
accomplish it in UI plz feel free to suggest.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-27 Thread mraliagha
Github user mraliagha commented on the issue:

https://github.com/apache/metron/pull/620
  
@iraghumitra Have you found any way to manage multiple Elasticsearch 
endpoints for load-balancing?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-25 Thread nickwallen
Github user nickwallen commented on the issue:

https://github.com/apache/metron/pull/620
  
@iraghumitra I really like what you've done.  There is a ton of 
functionality here and it works great.  I did not find any functional problems 
beyond what you've already called out.  I think we should get this into master 
and start iterating.

Also, thank you so much for summarizing the discussions that have already 
occurred in the PR description.  That was a HUGE help to me in reviewing.

I need to hold off on the +1 until we address a couple items.  
* [ ] 
[Licensing](https://github.com/apache/metron/pull/620#issuecomment-317834046)
* [ ] 
[Tests](https://github.com/apache/metron/pull/620#issuecomment-317788710) 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-25 Thread nickwallen
Github user nickwallen commented on the issue:

https://github.com/apache/metron/pull/620
  
Ran a license check on all dependencies, including transitive.

```
$ npm -g license-checker
$ cd metron-interface/metron-alerts
$ license-checker | grep licenses | sort | uniq -c | sort -n
   1├─ licenses: MIT
   1 │  ├─ licenses
   1 │  ├─ licenses: (BSD-2-Clause OR MIT OR Apache-2.0)
   1 │  ├─ licenses: (MIT AND CC-BY-3.0)
   1 │  ├─ licenses: (OFL-1.1 AND MIT)
   1 │  ├─ licenses: (WTFPL OR MIT)
   1 │  ├─ licenses: BSD-3-Clause OR MIT
   1 │  ├─ licenses: CC-BY-4.0
   1 │  ├─ licenses: Custom: 
https://github.com/substack/node-browserify
   1 │  ├─ licenses: LGPL
   1 │  ├─ licenses: LGPL-2.1+
   1 │  ├─ licenses: Public Domain
   1 │  ├─ licenses: UNLICENSED
   2 │  ├─ licenses: Apache License, Version 2.0
   2 │  ├─ licenses: Unlicense
   3 │  ├─ licenses: BSD
   7 │  ├─ licenses: BSD-2-Clause
  16 │  ├─ licenses: BSD*
  19 │  ├─ licenses: MIT*
  20 │  ├─ licenses: Apache-2.0
  24 │  ├─ licenses: BSD-3-Clause
  88 │  ├─ licenses: ISC
 762 │  ├─ licenses: MIT
```

The following licenses we need to follow-up on to see if they are 
permissible.

* `CC-BY-4.0`: https://www.apache.org/legal/resolved.html#category-b
```
├─ caniuse-db@1.0.3706
│  ├─ licenses: CC-BY-4.0
│  ├─ repository: https://github.com/Fyrd/caniuse
│  ├─ publisher: Alexis Deveria
│  ├─ email: adeve...@gmail.com
│  └─ licenseFile: 
/Users/nallen/tmp/metron-pr620/metron-interface/metron-alerts/node_modules/caniuse-db/LICENSE
```

* `LGPL`, `LGPL-2.1+`: https://www.apache.org/legal/resolved.html#category-x
```
├─ xmldom@0.1.27
│  ├─ licenses: LGPL
│  ├─ repository: https://github.com/jindw/xmldom
│  ├─ publisher: jindw
│  ├─ email: ji...@xidea.org
│  ├─ url: http://www.xidea.org
│  └─ licenseFile: 
/Users/nallen/tmp/metron-pr620/metron-interface/metron-alerts/node_modules/xmldom/LICENSE
```
```
├─ jschardet@1.5.0
│  ├─ licenses: LGPL-2.1+
│  ├─ repository: https://github.com/aadsm/jschardet
│  ├─ publisher: António Afonso
│  └─ licenseFile: 
/Users/nallen/tmp/metron-pr620/metron-interface/metron-alerts/node_modules/jschardet/LICENSE
```

* `Custom` @ https://github.com/substack/node-browserify
```
├─ weak-map@1.0.5
│  ├─ licenses: Custom: https://github.com/substack/node-browserify
│  ├─ repository: https://github.com/drses/weak-map
│  ├─ publisher: Mark Miller
│  ├─ email: erig...@gmail.com
│  └─ licenseFile: 
/Users/nallen/tmp/metron-pr620/metron-interface/metron-alerts/node_modules/weak-map/README.md
```

* `UNLICENSED`, `Unlicense`: ?
```
├─ tweetnacl@0.14.5
│  ├─ licenses: Unlicense
│  ├─ repository: https://github.com/dchest/tweetnacl-js
│  ├─ publisher: TweetNaCl-js contributors
│  └─ licenseFile: 
/Users/nallen/tmp/metron-pr620/metron-interface/metron-alerts/node_modules/tweetnacl/LICENSE
```
```
├─ spdx-license-ids@1.2.2
│  ├─ licenses: Unlicense
│  ├─ repository: https://github.com/shinnn/spdx-license-ids
│  ├─ publisher: Shinnosuke Watanabe
│  ├─ url: https://github.com/shinnn
│  └─ licenseFile: 
/Users/nallen/tmp/metron-pr620/metron-interface/metron-alerts/node_modules/spdx-license-ids/LICENSE
```

* Something must be wrong with our package definition.  We should indicate 
Apache for `metron-alerts`.
```
├─ metron-alerts@0.4.0
│  ├─ licenses: UNLICENSED
│  ├─ private: true
│  └─ licenseFile: 
/Users/nallen/tmp/metron-pr620/metron-interface/metron-alerts/README.md
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-25 Thread nickwallen
Github user nickwallen commented on the issue:

https://github.com/apache/metron/pull/620
  
It seems the end-to-end tests are failing for me.  This is what I did to 
run them.  The README is not totally clear to me on how these should be run.

1. `cd incubator-metron/metron-interface/metron-alerts`
1. `npm install`
1. `./scripts/start-dev.sh`
1. Open new console...
1. `cd incubator-metron/metron-interface/metron-alerts
1. `sh ./scripts/start-server-for-e2e.sh`
1. Open new console...
1. `cd incubator-metron/metron-interface/metron-alerts`
1. `npm run e2e`

And here is the output.
```
Spec started

  metron-alerts App
✗ should have all the UI elements
  - Expected 'Alerts (0 of )' to equal 'Alerts (25 of 169)' for alerts 
title
✗ should have all pagination controls and they should be working
  - Failed: No element found using locator: By(css selector, 
metron-table-pagination .fa.fa-chevron-left)
✓ should have all settings controls and they should be working
✓ play pause should start polling and stop polling
✗ should select columns from table configuration
  - Expected [ 'score' ] to equal [ 'score', '_id', 'timestamp', 
'source:type', 'ip_src_addr', 'enrichments:geo:ip_dst_addr:country', 
'ip_dst_addr', 'host', 'alert_status' ].
  - Failed: No element found using locator: By(css selector, 
app-configure-table label[for="select-deselect-_id"])

A Jasmine spec timed out. Resetting the WebDriver Control Flow.
  metron-alerts configure table
✗ should select columns from table configuration
  - Error: Timeout - Async callback was not invoked within timeout 
specified by jasmine.DEFAULT_TIMEOUT_INTERVAL.
  - Expected [ 'score' ] to equal [ 'score', '_id', 'timestamp', 
'source:type', 'ip_src_addr', 'enrichments:geo:ip_dst_addr:country', 
'ip_dst_addr', 'host', 'alert_status' ].
  - Failed: No element found using locator: By(css selector, 
app-configure-table label[for="select-deselect-_id"])

  metron-alerts Search
✓ should display all the default values for saved searches
✓ should have all save search controls and they save search should be 
working
✗ should populate search items when selected on table
  - Failed: Index out of bound. Trying to access element at index: 0, 
but there are only 0 elements that match locator By(link text, US)
✗ should delete search items from search box
  - Failed: Index out of bound. Trying to access element at index: 0, 
but there are only 0 elements that match locator By(link text, US)
✗ should delete first search items from search box having multiple 
search fields
  - Failed: Index out of bound. Trying to access element at index: 0, 
but there are only 0 elements that match locator By(link text, US)
✗ manually entering search queries to search box and pressing enter 
key should search
  - Failed: No element found using locator: By(css selector, 
metron-table-pagination span)

**
*Failures*
**

1) metron-alerts App should have all the UI elements
  - Expected 'Alerts (0 of )' to equal 'Alerts (25 of 169)' for alerts title

2) metron-alerts App should have all pagination controls and they should be 
working
  - Failed: No element found using locator: By(css selector, 
metron-table-pagination .fa.fa-chevron-left)

3) metron-alerts App should select columns from table configuration
  - Expected [ 'score' ] to equal [ 'score', '_id', 'timestamp', 
'source:type', 'ip_src_addr', 'enrichments:geo:ip_dst_addr:country', 
'ip_dst_addr', 'host', 'alert_status' ].
  - Failed: No element found using locator: By(css selector, 
app-configure-table label[for="select-deselect-_id"])

4) metron-alerts configure table should select columns from table 
configuration
  - Error: Timeout - Async callback was not invoked within timeout 
specified by jasmine.DEFAULT_TIMEOUT_INTERVAL.
  - Expected [ 'score' ] to equal [ 'score', '_id', 'timestamp', 
'source:type', 'ip_src_addr', 'enrichments:geo:ip_dst_addr:country', 
'ip_dst_addr', 'host', 'alert_status' ].
  - Failed: No element found using locator: By(css selector, 
app-configure-table label[for="select-deselect-_id"])

5) metron-alerts Search should populate search items when selected on table
  - Failed: Index out of bound. Trying to access element at index: 0, but 
there are only 0 elements that match locator By(link text, US)

6) metron-alerts Search should delete search items from search box
  - Failed: Index out of bound. Trying to access element at index: 0, but 
there are only 0 elements that match locator By(link 

[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-25 Thread BlackHatJohnDoe
Github user BlackHatJohnDoe commented on the issue:

https://github.com/apache/metron/pull/620
  
When doing an `npm install` I got:
```
node-pre-gyp info check checked for 
"/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/fsevents/lib/binding/Release/node-v57-darwin-x64/fse.node"
 (not found)
node-pre-gyp http GET 
https://fsevents-binaries.s3-us-west-2.amazonaws.com/v1.1.1/fse-v1.1.1-node-v57-darwin-x64.tar.gz
node-pre-gyp http 404 
https://fsevents-binaries.s3-us-west-2.amazonaws.com/v1.1.1/fse-v1.1.1-node-v57-darwin-x64.tar.gz
node-pre-gyp ERR! Tried to download(404): 
https://fsevents-binaries.s3-us-west-2.amazonaws.com/v1.1.1/fse-v1.1.1-node-v57-darwin-x64.tar.gz
node-pre-gyp ERR! Pre-built binaries not found for fsevents@1.1.1 and 
node@8.2.1 (node-v57 ABI) (falling back to source compile with node-gyp)
node-pre-gyp http 404 status code downloading tarball 
https://fsevents-binaries.s3-us-west-2.amazonaws.com/v1.1.1/fse-v1.1.1-node-v57-darwin-x64.tar.gz
```

After the install, an `npm ls` showed:
```
npm ERR! invalid: @angular/cli@1.0.0 
/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/@angular/cli
npm ERR! extraneous: zone.js@0.7.8 
/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/@angular/cli/node_modules/zone.js
npm ERR! extraneous: @ngtools/json-schema@1.0.5 
/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/@ngtools/json-schema
npm ERR! extraneous: @ngtools/webpack@1.3.0 
/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/@ngtools/webpack
npm ERR! extraneous: autoprefixer@6.7.7 
/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/autoprefixer
npm ERR! extraneous: common-tags@1.4.0 
/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/common-tags
npm ERR! extraneous: css-loader@0.26.4 
/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/css-loader
npm ERR! extraneous: cssnano@3.10.0 
/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/cssnano
npm ERR! extraneous: denodeify@1.2.1 
/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/denodeify
npm ERR! extraneous: ember-cli-normalize-entity-name@1.0.0 
/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/ember-cli-normalize-entity-name
npm ERR! extraneous: ember-cli-string-utils@1.1.0 
/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/ember-cli-string-utils
npm ERR! extraneous: exports-loader@0.6.4 
/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/exports-loader
npm ERR! extraneous: extract-text-webpack-plugin@2.0.0 
/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/extract-text-webpack-plugin
npm ERR! extraneous: file-loader@0.10.1 
/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/file-loader
npm ERR! extraneous: fs-extra@2.1.2 
/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/fs-extra
npm ERR! extraneous: html-webpack-plugin@2.28.0 
/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/html-webpack-plugin
npm ERR! extraneous: inflection@1.12.0 
/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/inflection
npm ERR! extraneous: inquirer@3.1.0 
/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/inquirer
npm ERR! extraneous: istanbul-instrumenter-loader@2.0.0 
/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/istanbul-instrumenter-loader
npm ERR! extraneous: json-loader@0.5.4 
/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/json-loader
npm ERR! extraneous: karma-sourcemap-loader@0.3.7 
/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/karma-sourcemap-loader
npm ERR! extraneous: karma-webpack@2.0.3 
/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/karma-webpack
npm ERR! extraneous: less@2.7.2 
/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/less
npm ERR! extraneous: less-loader@2.2.3 
/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/less-loader
npm ERR! extraneous: node-modules-path@1.0.1 
/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/node-modules-path
npm ERR! extraneous: nopt@4.0.1 
/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/nopt
npm ERR! extraneous: opn@4.0.2 
/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/opn
npm ERR! extraneous: portfinder@1.0.13 
/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/portfinder
npm ERR! extraneous: postcss-loader@0.13.0 
/Users/johndoe/dev/metron/metron-interface/metron-alerts/node_modules/postcss-loader
npm ERR! extraneous: postcss-url@5.1.2 

[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-25 Thread nickwallen
Github user nickwallen commented on the issue:

https://github.com/apache/metron/pull/620
  
Thanks @iraghumitra .  That fixed the build.  Will continue taking a look 
at it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-25 Thread iraghumitra
Github user iraghumitra commented on the issue:

https://github.com/apache/metron/pull/620
  
@nickwallen I fixed the build issue. Looks like the package-lock.json 
didn't do its job I will check it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-24 Thread nickwallen
Github user nickwallen commented on the issue:

https://github.com/apache/metron/pull/620
  
@BlackHatJohnDoe Thanks.  

Would you be able to run the following and share the output?  I'd like to 
compare what I have on a Mac versus what is working for you.

```
cd metron-interface/metron-alerts
npm ls
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-24 Thread BlackHatJohnDoe
Github user BlackHatJohnDoe commented on the issue:

https://github.com/apache/metron/pull/620
  
@nickwallen I had similar issues on my macOS system but when I went to 
CentOS it worked as is just fine.  I didn't go back and revisit - some brief 
research showed some versioning issues with dependencies.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-24 Thread nickwallen
Github user nickwallen commented on the issue:

https://github.com/apache/metron/pull/620
  
Did anything else change around dependencies that need installed?  I cannot 
get the `metron-alerts` project to build.  The only reference I see is Node > 
7.8.

```
$ mvn clean install 
...

[INFO] ERROR in ./src/main.ts
[INFO] Module not found: Error: Can't resolve 
'./$$_gendir/app/app.module.ngfactory' in 
'/Users/nallen/tmp/metron-pr620/metron-interface/metron-alerts/src'
[INFO]  @ ./src/main.ts 21:0-74
[INFO]  @ multi ./src/main.ts
[INFO]
[INFO] ERROR in ./src/$$_gendir async
[INFO] Module not found: Error: Can't resolve 
'/Users/nallen/tmp/metron-pr620/metron-interface/metron-alerts/src/$$_gendir/app/alerts/alerts-list/alerts-list.module.ngfactory.ts'
 in 
'/Users/nallen/tmp/metron-pr620/metron-interface/metron-alerts/src/$$_gendir'
[INFO]  @ ./src/$$_gendir async
[INFO]  @ ./~/@angular/core/@angular/core.es5.js
[INFO]  @ ./src/main.ts
[INFO]  @ multi ./src/main.ts
[INFO]
[INFO] ERROR in ./src/$$_gendir async
[INFO] Module not found: Error: Can't resolve 
'/Users/nallen/tmp/metron-pr620/metron-interface/metron-alerts/src/$$_gendir/app/alerts/save-search/save-search.module.ngfactory.ts'
 in 
'/Users/nallen/tmp/metron-pr620/metron-interface/metron-alerts/src/$$_gendir'
[INFO]  @ ./src/$$_gendir async
[INFO]  @ ./~/@angular/core/@angular/core.es5.js
[INFO]  @ ./src/main.ts
[INFO]  @ multi ./src/main.ts
[INFO]
[INFO] ERROR in ./src/$$_gendir async
[INFO] Module not found: Error: Can't resolve 
'/Users/nallen/tmp/metron-pr620/metron-interface/metron-alerts/src/$$_gendir/app/alerts/saved-searches/saved-searches.module.ngfactory.ts'
 in 
'/Users/nallen/tmp/metron-pr620/metron-interface/metron-alerts/src/$$_gendir'
[INFO]  @ ./src/$$_gendir async
[INFO]  @ ./~/@angular/core/@angular/core.es5.js
[INFO]  @ ./src/main.ts
[INFO]  @ multi ./src/main.ts
[ERROR]
[ERROR] npm ERR! Darwin 16.7.0
[ERROR] npm ERR! argv 
"/Users/nallen/tmp/metron-pr620/metron-interface/metron-alerts/node/node" 
"/Users/nallen/tmp/metron-pr620/metron-interface/metron-alerts/node/node_modules/npm/bin/npm-cli.js"
 "run" "build"
[ERROR] npm ERR! node v7.10.0
[ERROR] npm ERR! npm  v4.2.0
[ERROR] npm ERR! code ELIFECYCLE
[ERROR] npm ERR! errno 1
[ERROR] npm ERR! metron-alerts@0.4.0 build: 
`./node_modules/@angular/cli/bin/ng build -prod`
[ERROR] npm ERR! Exit status 1
[ERROR] npm ERR!
[ERROR] npm ERR! Failed at the metron-alerts@0.4.0 build script 
'./node_modules/@angular/cli/bin/ng build -prod'.
[ERROR] npm ERR! Make sure you have the latest version of node.js and npm 
installed.
[ERROR] npm ERR! If you do, this is most likely a problem with the 
metron-alerts package,
[ERROR] npm ERR! not with npm itself.
[ERROR] npm ERR! Tell the author that this fails on your system:
[ERROR] npm ERR! ./node_modules/@angular/cli/bin/ng build -prod
[ERROR] npm ERR! You can get information on how to open an issue for this 
project with:
[ERROR] npm ERR! npm bugs metron-alerts
[ERROR] npm ERR! Or if that isn't available, you can get their info via:
[ERROR] npm ERR! npm owner ls metron-alerts
[ERROR] npm ERR! There is likely additional logging output above.
[ERROR]
[ERROR] npm ERR! Please include the following file with any support request:
[ERROR] npm ERR! 
/Users/nallen/.npm/_logs/2017-07-24T18_40_10_074Z-debug.log
```

Cracking open the debug log referenced gives me this.
```
0 info it worked if it ends with ok
1 verbose cli [ 
'/Users/nallen/tmp/metron-pr620/metron-interface/metron-alerts/node/node',
1 verbose cli   
'/Users/nallen/tmp/metron-pr620/metron-interface/metron-alerts/node/node_modules/npm/bin/npm-cli.js',
1 verbose cli   'run',
1 verbose cli   'build' ]
2 info using npm@4.2.0
3 info using node@v7.10.0
4 verbose run-script [ 'prebuild', 'build', 'postbuild' ]
5 info lifecycle metron-alerts@0.4.0~prebuild: metron-alerts@0.4.0
6 silly lifecycle metron-alerts@0.4.0~prebuild: no script for prebuild, 
continuing
7 info lifecycle metron-alerts@0.4.0~build: metron-alerts@0.4.0
8 verbose lifecycle metron-alerts@0.4.0~build: unsafe-perm in lifecycle true
9 verbose lifecycle metron-alerts@0.4.0~build: PATH: 
/Users/nallen/tmp/metron-pr620/metron-interface/metron-alerts/node/node_modules/npm/bin/node-gyp-bin:/Users/nallen/tmp/metron-pr620/metron-interface/metron-alerts/node_modules/.bin:/Users/nallen/tmp/metron-pr620/metron-interface/metron-alerts/node:/usr/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Applications/Wireshark.app/Contents/MacOS:/Users/nallen/bin:/Users/nallen/bin:/usr/local/Cellar/kafka/0.10.1.1//bin:/usr/local/Cellar/apache-spark/2.1.1/libexec/bin:
10 verbose lifecycle metron-alerts@0.4.0~build: CWD: 

[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-13 Thread mraliagha
Github user mraliagha commented on the issue:

https://github.com/apache/metron/pull/620
  
@iraghumitra You're welcome.

As a load balancer. For Elasticsearch client, if you provide a list of 
endpoints, it acts as a load balancer to make sure one of them will not get 
overwhelmed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-13 Thread iraghumitra
Github user iraghumitra commented on the issue:

https://github.com/apache/metron/pull/620
  
@mraliagha Glad, it worked and thanks for persisting with the issue. 

As for supporting multiple Elasticsearch urls. Are you looking for a load 
balancer between two ES that has the same data or the two ES instances have 
different data ?.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-13 Thread iraghumitra
Github user iraghumitra commented on the issue:

https://github.com/apache/metron/pull/620
  
@mraliagha I can get them from ES as I mentioned in my earlier comments, 
there was a bug in the code that was displaying only the fields names from bro 
indexes. It is fixed in this [commit](046c2ef4305b6317c8f0378e737f0e50af41c76a) 
. If you can check the fix on the latest code base it would be of great help.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-12 Thread mraliagha
Github user mraliagha commented on the issue:

https://github.com/apache/metron/pull/620
  
@iraghumitra We are using ASA and CEF parsers. Can't you get the field 
names dynamically from Elasticsearch?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-11 Thread mraliagha
Github user mraliagha commented on the issue:

https://github.com/apache/metron/pull/620
  
@iraghumitra I've tested your latest commit and it is much better now. 
However, I cannot see all the fields in the customise visible fields panel.


![image](https://user-images.githubusercontent.com/8438293/28059526-dcb212a2-6667-11e7-94b1-9e65eee43be5.png)



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-10 Thread mraliagha
Github user mraliagha commented on the issue:

https://github.com/apache/metron/pull/620
  
@iraghumitra I am going to test your latest code to make sure the mentioned 
issues haven't been resolved yet. I was using an older version of your build, 
so it might be outdated.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-10 Thread merrimanr
Github user merrimanr commented on the issue:

https://github.com/apache/metron/pull/620
  
I just added some comments related to abstracting the search service.  Much 
improved with the latest commits.  I think we're almost there.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-10 Thread iraghumitra
Github user iraghumitra commented on the issue:

https://github.com/apache/metron/pull/620
  
@mraliagha 
 - For 1 can you check with the latest code base. I am unable to simulate 
this. 
-  For 2 I am slightly confused here. I wanted to know the result of the 
GET rest call` 
http://:/_cluster/state`. 
This call fetches all the fields from elastic. All the default fields are added 
via Javascript so the rest call showing just the default fields is not 
possible. 
The rest call should either show all fields or show an error. Am I missing 
anything here?  You can fire a get call from browser to check the result.

Thanks for taking the time to review :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-09 Thread iraghumitra
Github user iraghumitra commented on the issue:

https://github.com/apache/metron/pull/620
  
@mraliagha added the fix suggested in "Retrieves all fields related to 
events". For rest of the issues, I am happy to work on them if you can help me 
with the data I need.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-06 Thread iraghumitra
Github user iraghumitra commented on the issue:

https://github.com/apache/metron/pull/620
  
@mraliagha
 - Do you see a play icon on the UI or a pause icon also can you share the 
refresh interval that is set. You can click on the sliders icon before 
paly/pause button to see this
 - For listing available fields I am using  '_cluster/state' API to 
retrieve list of field names. Does this api show all your fields? If not we can 
discuss this more
 - Score was incorrectly mapped I realized it late and fixed it in this 
[commit](https://github.com/apache/metron/pull/620/commits/91b41ae924f76d2f95dbe4748cfd5040f5db9328).
 The latest code should have a fix for it. The score is a read-only field the 
idea here is since the score is a critical field to understand the severity of 
the event we wanted to show it all the time, this is by design we can discuss 
this I am flexible on this.
 - "Retrieves all fields related to events" makes total sense let me see if 
we can quickly implement this




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-06 Thread mraliagha
Github user mraliagha commented on the issue:

https://github.com/apache/metron/pull/620
  
@iraghumitra I cannot see any error in JS console, except the following 
warning which I don't think is really important.
Angular is running in the development mode. Call enableProdMode() to enable 
the production mode.

More details for the customise functionality for the visible fields:
- I can see only a few number of available fields not all of them in that 
window. 
- "Score" field doesn't do anything. I tried to remove it, and it didn't 
work. It doesn't represent the treat triage score. It seems it is just a hard 
coded field.

Regarding the " retrieves all fields related to events" issue that I have 
mentioned, no I meant it is retrieving all fields not all of the events. So in 
RDBMS terminology, it is like "select *" right now instead of selecting only 
those fields that are visible in GUI. It is affecting query fetch time 
significantly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-06 Thread iraghumitra
Github user iraghumitra commented on the issue:

https://github.com/apache/metron/pull/620
  
@mraliagha Can you send me the screenshots for the below two issues and 
also are you seeing any errors in js console ?
 - The play/pause button doesn't work properly and creates and infinite 
loop sometimes which act as a DOS attack on Elasticsearch.
 - The customise functionality for managing visible fields in UI doesn't 
work.

For the other issues 
 - Alert-UI configuration only accepts a single URL for Elasticsearch 
endpoint. It doesn't accept a list of URLs.  _I will try to figure out a way to 
do this_
- The Alert-UI search functionality retrieves all fields related to events 
rather than only target the feasible ones. _You mean the UI should retrieve 
only the events with is_alerts true, then your observation is right. We can add 
is_alerts as a global search criterion under settings pane._



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-05 Thread mraliagha
Github user mraliagha commented on the issue:

https://github.com/apache/metron/pull/620
  
Hi,

There are few issues that we have faced during our testing. However, I am 
not sure they have been fixed already or not. I am just going to note them.

- The play/pause button doesn't work properly and creates and infinite loop 
sometimes which act as a DOS attack on Elasticsearch. 
- Alert-UI configuration only accepts a single URL for Elasticsearch 
endpoint. It doesn't accept a list of URLs.
- The customise functionality for managing visible fields in UI doesn't 
work.
- The Alert-UI search functionality retrieves all fields related to events 
rather than only target the feasible ones.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-03 Thread iraghumitra
Github user iraghumitra commented on the issue:

https://github.com/apache/metron/pull/620
  
@cestella & @merrimanr I have abstracted all the API calls to 
data-source.ts this should simplify plugging in any API provider.

@merrimanr plz let me know if search api looks better in terms of typings. 
Now you can look at abstract class 'DataSource' and 
'ElasticSearchLocalstorageImpl' to review all the API calls.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-02 Thread iraghumitra
Github user iraghumitra commented on the issue:

https://github.com/apache/metron/pull/620
  
@merrimanr QueryBuilder is thought of as a model than a utils class, the 
fields with '_' are used for display purpose and are not required to be 
persisted. I will take an another pass to see if I can refactor the code to 
make it look simple.In fact, the Saved Search and Recent Search Saves the query 
builder.
I understand that SearchRequest might be a more suitable name so renaming 
QueryBuilder to SearchRequest makes sense ?. I am taking an another pass at 
abstracting all the API calls so that we can change the api provider to 'solr' 
or rest api easily form the UI. Once i do that the service classes would look 
better.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-06-30 Thread iraghumitra
Github user iraghumitra commented on the issue:

https://github.com/apache/metron/pull/620
  
@merrimanr sounds good I will map the field's as they are today. Can you 
take another pass at it and let me know if it looks good.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-06-29 Thread merrimanr
Github user merrimanr commented on the issue:

https://github.com/apache/metron/pull/620
  
Here's an idea to get us started.  A search request could look like:
```
{
  "query": "ip_src_addr:192.168.66.1",
  "from": 0,
  "size": 25,
  "sort": [
{ "timestamp": "desc"},
{ "some_other_field": "asc"}
  ]
}
```
where the query field is a lucene-style query (which is supported by both 
ES and Solr I believe).  A search response could look like:
```
{
  "total": 1,
  "results":
[
  {
"id": "1234abcd",
"score": 1.0,
"source": {
  "source:type": "bro",
  "host": "some host",
  "ip_src_addr": "192.168.66.1"
}
  },
  {
"id": "5678efgh",
"score": 1.0,
"source": {
  "source:type": "snort",
  "ip_src_port": 8080,
  "ip_src_addr": "192.168.66.1"
}
  }
  ]
}
```

We can add other fields (aggregations for example) as we add more features. 
 What do you think?  Is this enough to drive the current alerts UI or are there 
more fields needed?  Would it be helpful to add a 'filters' field to the 
request or can we assume the client-side code will handle adding filters to the 
lucene query?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-06-29 Thread merrimanr
Github user merrimanr commented on the issue:

https://github.com/apache/metron/pull/620
  
If you want to make it specific to 'Alert' then I'm fine with that.  I'm 
not crazy about 'AlertsDTO', would prefer to keep the search result context in 
the name, something like 'AlertsSearchResponse'.  The actual name is probably 
not as important though.

QueryBuilder is fine but the getESSearchQuery should return a SearchRequest 
(or similarly named type).  Also, you have QueryBuilder under 'model' and I 
don't think that is accurate.  How about moving the fields in QueryBuilder 
(from, size, sort, etc) to some kind of SearchRequest model class and make 
QueryBuilder responsible for building a SearchRequest?  SearchRequest is a 
model object that just contains fields and QueryBuilder takes care of query 
(SearchRequest) generation.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-06-28 Thread merrimanr
Github user merrimanr commented on the issue:

https://github.com/apache/metron/pull/620
  
Just tested again and I am able to now remove the first filter and properly 
filter on values with special characters (referrer field for example).  I did 
another pass and found some trivial issues as well as a few non-trivial issues 
and have made comments.

I think more thought needs to be put into the AlertService.search and 
AlertService.pollSearch functions.  The AlertService.getAlert function is very 
clear to me:  it requires a couple of clearly named parameters and I expect to 
get an 'Alert' type object back.  The other functions in this service are not 
as clear.  The search function for example takes in a QueryBuilder object which 
provides a generic javascript object as the body for the post request.  Then in 
return the post returns an Observable with a generic javascript object.  So 
essentially Typescript isn't being used here when it should because it would 
make the search interface clearer.

 For example, I would prefer this function signature:
`public search(searchRequest: SearchRequest): Observable`

where SearchRequest and SearchResponse are model objects.  The way it is 
now it's not easy to understand what is being sent and what is expected back 
unless you've spent time tracing the search calls to where requests are 
built/response are processed and know all the source code well OR already has a 
lot of experience with the ES query syntax.

The result of all this is that not having a clear contract between the 
search client/server will make developing a middle-tier more tedious.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-06-21 Thread iraghumitra
Github user iraghumitra commented on the issue:

https://github.com/apache/metron/pull/620
  
@merrimanr I missed escaping values for the search request. I fixed it now, 
the 'referer' search should work now. Supporting 'OR' and 'NOT' operators in 
search request needs some work is it fine if I raise a JIRA and take some time 
to work on it. I updated the PR summary to point out that OR and NOT or not 
supported as of now.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-06-20 Thread iraghumitra
Github user iraghumitra commented on the issue:

https://github.com/apache/metron/pull/620
  
@merrimanr thanks for pointing me in the right direction.
@cestella I update the 'Contributor Comments' with all the required 
information. Please let me know if I am missing anything


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-06-20 Thread merrimanr
Github user merrimanr commented on the issue:

https://github.com/apache/metron/pull/620
  
Ok I will assume anything related to Alert Status is a not-yet-functional 
portion.

I am testing in full dev.  For the 2 you are not able to reproduce:

- Add the "referrer" field to your table.  Then filter on a value in that 
column that starts with "http://; and you should get an "all shards failed" 
error.
- The "OR" query will appear to be working but it's not bringing back the 
correct results.  The results should only contains values in the OR terms but 
it brings back all values.  It looks like an extra \  is being added to the 
query for some reason.  I've seen this problem in other queries too.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-06-20 Thread iraghumitra
Github user iraghumitra commented on the issue:

https://github.com/apache/metron/pull/620
  
@merrimanr thanks for taking time to validate the UI
 - This needs a fix
 - Alert Status is a UI field that can be used to integrate with an 
external ticketing system. This is just an example as of now to show the 
usefulness of the system. This can be a configurable option  in rest api
 - Same as above
 - I felt we should not have fixed widths for a table since the table can 
have a lot of data currently it auto adjusts. If this is causing jarring 
experience we can fix the widths of the columns
 - It is sorting only the first time, will check this
 - Can you give some more info on this, this seems to be working fine for me
 - Same as point 2
 - This is working fine. The OR query syntax is a valid ES syntax. Do you 
see any issue with the query?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-06-19 Thread merrimanr
Github user merrimanr commented on the issue:

https://github.com/apache/metron/pull/620
  
I was able to get this running in "dev" mode against full dev based on 
instructions in the README.  This is a great start.

I've noticed several bugs while initially exploring the UI:

- When multiple filters are applied, I am not able to successfully remove 
the first one.  For example given a query "source:type:snort AND 
ip_dst_addr:192.168.138.158", when I remove the source:type filter it becomes 
"AND ip_dst_addr:192.168.138.158" and I get a failed to parse error.
- Filtering by alert status always returns 0 records
- What is supposed to happen when you change an alert status?  Right now it 
looks like nothing happens.
- Changing the sort order resizes all the columns more than it should.
- _id column sorting doesn't do anything.
- Filtering on various columns causes an error ("url" and "referrer" for 
example).
- Does the ACTIONS dropdown do anything right now?
- OR operators don't seem to work.  For example 
"ip_src_addr:192.168.138.158 OR ip_src_addr:192.168.66.1" returns records with 
ip_src_addr that does not match those filters (actual query sent ends up being 
"query": "ip_src_addr\\:192.168.138.158 OR ip_src_addr:192.168.66.1").

Will report more as I find them.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-06-19 Thread cestella
Github user cestella commented on the issue:

https://github.com/apache/metron/pull/620
  
So, given that we're planning on moving to the REST API to enable 
index-specific capabilities, can you detail exactly which API calls that this 
PR will require?  We can then create JIRAs.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-06-19 Thread iraghumitra
Github user iraghumitra commented on the issue:

https://github.com/apache/metron/pull/620
  
@cestella thanks for pointing me to PR #468 this is great. I will update 
the PR with all the steps ASAIC. Meanwhile, if you want to give it a spin the 
Readme has all the required info. 'Development Setup' or 'Installing on an 
existing Cluster' along with 'Prerequisites' would be a good starting point.

@ottobackwards The GUI works with its own models abstracting it from ES or 
Solr. These models are populated in the service's and service's fetch data from 
ES and local storage for now. The services can be easily modified to talk to 
REST based service. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-06-19 Thread cestella
Github user cestella commented on the issue:

https://github.com/apache/metron/pull/620
  
@ottobackwards yes, that's exactly what I mean.  I want to make sure this 
work is teed up to easily use the rest-api.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-06-19 Thread ottobackwards
Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/620
  
I think and hope what @cestella is saying is that we would hope that the 
metron ui's are not tied to an optional back end, but rather to the rest-api.

Thus Metron alerts will work with SOLR etc.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-06-19 Thread cestella
Github user cestella commented on the issue:

https://github.com/apache/metron/pull/620
  
I think it's ok to not integrate with REST for the first cut, but I'd like 
to see the UI layer architected in such a way that when we do integrate with 
the REST layer, it'll not span more than modifications of service classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---