[
https://issues.apache.org/jira/browse/METRON-1554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583967#comment-16583967
]
ASF GitHub Bot commented on METRON-1554:
----------------------------------------
GitHub user merrimanr opened a pull request:
https://github.com/apache/metron/pull/1169
METRON-1554: Pcap Query Panel
## Contributor Comments
Per the discussion
[here](https://lists.apache.org/thread.html/94ebc9be23f6f2ec8c53f8f6b71e97d6919baf415caf534e2b25ba9b@%3Cdev.metron.apache.org%3E),
we have several votes for merging this feature branch into master.
All of the tasks included in this PR can be found in the original
[Jira](https://issues.apache.org/jira/browse/METRON-1554). It might be worth
running through a final test before we merge. Here are the items I would
suggest we include in testing:
- Run queries (with various filters applied) and verify results are display
in the UI
- Run a query and verify results are returned with the CLI
- Verify input validation is working for each UI filter input
- Verify the UI can be refreshed and the currently running query is still
displayed
- Verify pcap queries can be configured to run in a dedicated YARN queue
- Verify results can be paged through in the UI and downloaded as raw Pcap
data
- Verify proper error messages are returned (wireshark not installed, no
results returned, etc)
- Verify jobs can be killed in the UI
- Verify page size, YARN queue, and finalizer thread pool can be configured
in Ambari
## Pull Request Checklist
Thank you for submitting a contribution to Apache Metron.
Please refer to our [Development
Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235)
for the complete guide to follow for contributions.
Please refer also to our [Build Verification
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview)
for complete smoke testing guides.
In order to streamline the review of the contribution we ask you follow
these guidelines and ask you to double check the following:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to
be created at [Metron
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON-XXXX where XXXX is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically master)?
### For code changes:
- [x] Have you included steps to reproduce the behavior or problem that is
being changed or addressed?
- [x] Have you included steps or a guide to how the change may be verified
and tested manually?
- [x] Have you ensured that the full suite of tests and checks have been
executed in the root metron folder via:
```
mvn -q clean integration-test install &&
dev-utilities/build-utils/verify_licenses.sh
```
- [x] Have you written or updated unit tests and or integration tests to
verify your changes?
- [x] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [x] Have you verified the basic functionality of the build by building
and running locally with Vagrant full-dev environment or the equivalent?
### For documentation related changes:
- [x] Have you ensured that format looks appropriate for the output in
which it is rendered by building and verifying the site-book? If not then run
the following commands and the verify changes via
`site-book/target/site/index.html`:
```
cd site-book
mvn site
```
#### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
It is also recommended that [travis-ci](https://travis-ci.org) is set up
for your personal repository such that your branches are built there before
submitting a pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/merrimanr/incubator-metron
feature/METRON-1554-pcap-query-panel
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metron/pull/1169.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1169
----
commit 165c0e716342d743460123a9de6303178ed32830
Author: merrimanr <merrimanr@...>
Date: 2018-06-26T15:18:01Z
METRON-1555 Update REST to run YARN and MR jobs (merrimanr) closes
apache/metron#1019
commit 4e586ac553daed1eb8322b2bbc4d83fcbdfb89d8
Author: rmerriman <merrimanr@...>
Date: 2018-06-26T15:35:55Z
Merge branch 'master' into feature/METRON-1554-pcap-query-panel
commit 9cee51eb2c428eb7bf3aa791f5ec1293ed36c43a
Author: mmiklavc <michael.miklavcic@...>
Date: 2018-07-11T01:24:58Z
METRON-1641: Enable Pcap jobs to be submitted asynchronously (mmiklavc via
mmiklavc) closes apache/metron#1081
commit f30e9cc1fb9b6985e7ad7582c764d035b5cb0e1e
Author: Michael Miklavcic <michael.miklavcic@...>
Date: 2018-07-11T01:31:56Z
Resolve merge conflicts with master.
commit dbbf624367f8a3fb5d78352be0d9a2dec0533923
Author: mmiklavc <michael.miklavcic@...>
Date: 2018-07-18T21:48:08Z
METRON-1614: Create job status abstraction (mmiklavc via mmiklavc) closes
apache/metron#1108
commit 39ae9f4642073d3d4f0fa423339dd97f85974588
Author: merrimanr <merrimanr@...>
Date: 2018-07-19T16:01:49Z
METRON-1674 Create REST endpoint for job status abstraction (merrimanr)
closes apache/metron#1109
commit f316d15f082415f44d22bb124818ec116985ba1f
Author: mmiklavc <michael.miklavcic@...>
Date: 2018-07-19T21:36:12Z
METRON-1686 Create stop job endpoint for Pcap queries (mmiklavc via
merrimanr) closes apache/metron#1115
commit 3e5ef41d9b8639fb1155686e615c02a59b735397
Author: merrimanr <merrimanr@...>
Date: 2018-07-19T22:13:22Z
METRON-1638 Retrieve Pcap results in pdml format (merrimanr) closes
apache/metron#1120
commit a5a51399d2eafd2535d79bb13ee0d4d8eb2e2d23
Author: merrimanr <merrimanr@...>
Date: 2018-07-20T14:37:34Z
METRON-1685 Retrieve Pcap results in raw binary format (merrimanr) closes
apache/metron#1123
commit b580a69f1d9d3346171d749dcd6f791a9bc6d894
Author: Michael Miklavcic <michael.miklavcic@...>
Date: 2018-07-20T15:40:09Z
Merge branch 'master' into feature/METRON-1554-pcap-query-panel
commit 8763fba2ff579b37b6af82822fc4b2c4390665a8
Author: merrimanr <merrimanr@...>
Date: 2018-07-20T16:05:32Z
METRON-1562 Enable Kerberos in REST for YARN and MR jobs (merrimanr) closes
apache/metron#1094
commit e78bc530260aa622e8263cda2b94be99194dffb5
Author: merrimanr <merrimanr@...>
Date: 2018-07-23T14:08:12Z
METRON-1560 Update MPack to support Pcap panel (merrimanr) closes
apache/metron#1124
commit d0e87576944b6370eb1d9a0c98d0450f2c51f052
Author: merrimanr <merrimanr@...>
Date: 2018-07-23T14:09:37Z
METRON-1661 Create Pcap Query Filter endpoint (merrimanr) closes
apache/metron#1125
commit 9d4842f3d6f355d449d51f3c6ac02f2c904c295d
Author: mmiklavc <michael.miklavcic@...>
Date: 2018-07-25T14:21:32Z
METRON-1690: Add more context to PcapJob JobStatus (mmiklavc via mmiklavc)
closes apache/metron#1128
commit f1f5dda4c643394c12ab9f4501c5b806adb21129
Author: mmiklavc <michael.miklavcic@...>
Date: 2018-07-25T14:26:45Z
METRON-1693: Fix Pcap CLI local FS finalizer (mmiklavc via mmiklavc) closes
apache/metron#1130
commit 6c90724d8d4f06c453128f860eff51037f1870d8
Author: merrimanr <merrimanr@...>
Date: 2018-07-25T16:58:05Z
METRON-1691: REST should limit the number of Pcap jobs a user can submit
(merrimanr via mmiklavc) closes apache/metron#1129
commit 7dff4def23a7c839ea599a120ab6d4cf5b9bbbef
Author: cstella <cestella@...>
Date: 2018-07-26T16:29:04Z
Merge branch 'master' into feature/METRON-1554-pcap-query-panel
commit d5eb56a967ac7b925d0a843c027a2b0dde66de06
Author: tiborm <tibor.meller@...>
Date: 2018-08-01T16:31:50Z
METRON-1671 Create PCAP UI (tiborm via merrimanr) closes apache/metron#1103
commit 2b6959b46db79256d12b8522385212559cabada2
Author: merrimanr <merrimanr@...>
Date: 2018-08-02T17:17:37Z
METRON-1700 Create REST endpoint to get job configuration (merrimanr)
closes apache/metron#1135
commit b445bfe242123e544c1fef41e2ab5bdc07dbc56e
Author: tiborm <tibor.meller@...>
Date: 2018-08-03T13:13:11Z
METRON-1662 PCAP UI - Downloading PCAP page files (tiborm via merrimanr)
closes apache/metron#1118
commit 09a7f18994dc8254ca158139c44258cb7f1107ec
Author: tiborm <tibor.meller@...>
Date: 2018-08-06T14:14:54Z
METRON-1676 PCAP UI - Add data range selector to the filter bar (tiborm via
merrimanr) closes apache/metron#1119
commit 897266598b7984884d6cf8a6a699fe902e792483
Author: merrimanr <merrimanr@...>
Date: 2018-08-06T16:08:48Z
Merge branch 'master' into feature/METRON-1554-pcap-query-panel
commit 756ceedc3d3567f3508478eaa5ed29f12c22688a
Author: merrimanr <merrimanr@...>
Date: 2018-08-06T18:36:40Z
METRON-1721 New default input path is wrong in pcap CLI (merrimanr) closes
apache/metron#1137
commit 037b50b9a829d7292db1148fc628d9231da499e1
Author: sardell <sardell@...>
Date: 2018-08-07T14:04:33Z
METRON-1675 PCAP UI - Introduce the paging capability (sardell via
merrimanr) closes apache/metron#1121
commit 3e7785920e8f0c5aadf36e05dbcfa69252cc5574
Author: sardell <sardell@...>
Date: 2018-08-07T18:58:08Z
METRON-1683 PCAP UI - Fix the download progress bar (sardell via merrimanr)
closes apache/metron#1122
commit 7967f358c6c4b8437935c43e54179126e48e248f
Author: mmiklavc <michael.miklavcic@...>
Date: 2018-08-07T21:02:20Z
METRON-1726: Refactor PcapTopologyIntegrationTest (mmiklavc via mmiklavc)
closes apache/metron#1140
commit afb9607dfbbaf89f06a30823bf668523bb7b6f9a
Author: merrimanr <merrimanr@...>
Date: 2018-08-08T19:11:26Z
METRON-1720 Better error messages when there are no results or wireshark is
not installed (merrimanr) closes apache/metron#1154
commit 52de126fc746181d6aac8ce002d6b814fd1e6cb5
Author: tiborm <tibor.meller@...>
Date: 2018-08-08T21:36:35Z
METRON-1712 PCAP UI - Input validation (tiborm via merrimanr) closes
apache/metron#1142
commit 5b3e2c337b3772bcd7d18cedac75877bbe6f7735
Author: sardell <sardell@...>
Date: 2018-08-09T12:44:15Z
METRON-1723 PCAP UI - Unable to select/copy from packets details in PCAP
query panel (sardell via merrimanr) closes apache/metron#1139
commit 14dcb2d90581835d8206c65918c24e8cb04bfd06
Author: tiborm <tibor.meller@...>
Date: 2018-08-09T13:33:21Z
METRON-1713 PCAP UI - Add a way to kill a pcap job (tiborm via merrimanr)
closes apache/metron#1143
----
> Pcap Query Panel
> ----------------
>
> Key: METRON-1554
> URL: https://issues.apache.org/jira/browse/METRON-1554
> Project: Metron
> Issue Type: New Feature
> Reporter: Ryan Merriman
> Priority: Major
>
> Legacy OpenSOC included a panel in Kibana that allowed users to query for
> pcap data. We would like to add this feature back into Metron. There are 2
> discussions happening on the dev list where we are gathering user
> requirements:
> [http://mail-archives.apache.org/mod_mbox/metron-dev/201805.mbox/%3CCAEVkqPYxfe3Q65mX7Mkuk_FKUCV420yb6hcLmf+FF=1ozer...@mail.gmail.com%3E]
> and working through the backend architecture:
> [http://mail-archives.apache.org/mod_mbox/metron-dev/201805.mbox/%3ccaevkqpbxzjnu_wgrbfwnz-mvqnkb7mthedveq9plyhwfit7...@mail.gmail.com%3E]
> Forthcoming sub tasks will be based on the outcome of these discussions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
