[jira] [Commented] (METRON-1340) Improve e2e tests for metron alerts
[ https://issues.apache.org/jira/browse/METRON-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687123#comment-16687123 ] ASF GitHub Bot commented on METRON-1340: Github user gmcdonald closed the pull request at: https://github.com/apache/metron/pull/857 > Improve e2e tests for metron alerts > --- > > Key: METRON-1340 > URL: https://issues.apache.org/jira/browse/METRON-1340 > Project: Metron > Issue Type: Bug >Reporter: RaghuMitra >Assignee: RaghuMitra >Priority: Major > > Need to improve e2e tests in the following areas: > - Tests should not be flaky > - Remove the sleep ( This should implicitly make the tests run faster) > - Truncate HBase table 'metron_update' before starting the tests > - Improve the tests descriptions > - Run the tests headless if possible > - Check the node version and browser version before launching the tests > The expected behavior is that there are no intermittent failures. Acceptance > criteria: 5 consecutive runs without failures. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1340) Improve e2e tests for metron alerts
[ https://issues.apache.org/jira/browse/METRON-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687059#comment-16687059 ] ASF GitHub Bot commented on METRON-1340: Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/857 I also requested that this be closed in this ticket; https://issues.apache.org/jira/browse/INFRA-17251 > Improve e2e tests for metron alerts > --- > > Key: METRON-1340 > URL: https://issues.apache.org/jira/browse/METRON-1340 > Project: Metron > Issue Type: Bug >Reporter: RaghuMitra >Assignee: RaghuMitra >Priority: Major > > Need to improve e2e tests in the following areas: > - Tests should not be flaky > - Remove the sleep ( This should implicitly make the tests run faster) > - Truncate HBase table 'metron_update' before starting the tests > - Improve the tests descriptions > - Run the tests headless if possible > - Check the node version and browser version before launching the tests > The expected behavior is that there are no intermittent failures. Acceptance > criteria: 5 consecutive runs without failures. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1340) Improve e2e tests for metron alerts
[ https://issues.apache.org/jira/browse/METRON-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662326#comment-16662326 ] ASF GitHub Bot commented on METRON-1340: Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/857 Please close this > Improve e2e tests for metron alerts > --- > > Key: METRON-1340 > URL: https://issues.apache.org/jira/browse/METRON-1340 > Project: Metron > Issue Type: Bug >Reporter: RaghuMitra >Assignee: RaghuMitra >Priority: Major > > Need to improve e2e tests in the following areas: > - Tests should not be flaky > - Remove the sleep ( This should implicitly make the tests run faster) > - Truncate HBase table 'metron_update' before starting the tests > - Improve the tests descriptions > - Run the tests headless if possible > - Check the node version and browser version before launching the tests > The expected behavior is that there are no intermittent failures. Acceptance > criteria: 5 consecutive runs without failures. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1340) Improve e2e tests for metron alerts
[ https://issues.apache.org/jira/browse/METRON-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662327#comment-16662327 ] ASF GitHub Bot commented on METRON-1340: Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/857 please close this > Improve e2e tests for metron alerts > --- > > Key: METRON-1340 > URL: https://issues.apache.org/jira/browse/METRON-1340 > Project: Metron > Issue Type: Bug >Reporter: RaghuMitra >Assignee: RaghuMitra >Priority: Major > > Need to improve e2e tests in the following areas: > - Tests should not be flaky > - Remove the sleep ( This should implicitly make the tests run faster) > - Truncate HBase table 'metron_update' before starting the tests > - Improve the tests descriptions > - Run the tests headless if possible > - Check the node version and browser version before launching the tests > The expected behavior is that there are no intermittent failures. Acceptance > criteria: 5 consecutive runs without failures. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1340) Improve e2e tests for metron alerts
[ https://issues.apache.org/jira/browse/METRON-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366728#comment-16366728 ] ASF GitHub Bot commented on METRON-1340: Github user iraghumitra commented on the issue: https://github.com/apache/metron/pull/857 Got busy with too many things happening around me, will yield this a conclusion next week. > Improve e2e tests for metron alerts > --- > > Key: METRON-1340 > URL: https://issues.apache.org/jira/browse/METRON-1340 > Project: Metron > Issue Type: Bug >Reporter: RaghuMitra >Assignee: RaghuMitra >Priority: Major > > Need to improve e2e tests in the following areas: > - Tests should not be flaky > - Remove the sleep ( This should implicitly make the tests run faster) > - Truncate HBase table 'metron_update' before starting the tests > - Improve the tests descriptions > - Run the tests headless if possible > - Check the node version and browser version before launching the tests > The expected behavior is that there are no intermittent failures. Acceptance > criteria: 5 consecutive runs without failures. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1340) Improve e2e tests for metron alerts
[ https://issues.apache.org/jira/browse/METRON-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16308152#comment-16308152 ] ASF GitHub Bot commented on METRON-1340: Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/857 @justinleet I believe @iraghumitra is still working on the SELENIUM_PROMISE_MANAGER change proposed above but I will defer to him. > Improve e2e tests for metron alerts > --- > > Key: METRON-1340 > URL: https://issues.apache.org/jira/browse/METRON-1340 > Project: Metron > Issue Type: Bug >Reporter: RaghuMitra >Assignee: RaghuMitra > > Need to improve e2e tests in the following areas: > - Tests should not be flaky > - Remove the sleep ( This should implicitly make the tests run faster) > - Truncate HBase table 'metron_update' before starting the tests > - Improve the tests descriptions > - Run the tests headless if possible > - Check the node version and browser version before launching the tests > The expected behavior is that there are no intermittent failures. Acceptance > criteria: 5 consecutive runs without failures. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (METRON-1340) Improve e2e tests for metron alerts
[ https://issues.apache.org/jira/browse/METRON-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16308127#comment-16308127 ] ASF GitHub Bot commented on METRON-1340: Github user justinleet commented on the issue: https://github.com/apache/metron/pull/857 @merrimanr @mmiklavc @iraghumitra What are the next steps towards being able to have this stable? It sounds like there was some playing around with versions, and that there may be some more structural changes to the code style. How do you propose we move forward? I'd really like to see this stuff made stable and I'm happy to help out with more testing or whatever else we need. > Improve e2e tests for metron alerts > --- > > Key: METRON-1340 > URL: https://issues.apache.org/jira/browse/METRON-1340 > Project: Metron > Issue Type: Bug >Reporter: RaghuMitra >Assignee: RaghuMitra > > Need to improve e2e tests in the following areas: > - Tests should not be flaky > - Remove the sleep ( This should implicitly make the tests run faster) > - Truncate HBase table 'metron_update' before starting the tests > - Improve the tests descriptions > - Run the tests headless if possible > - Check the node version and browser version before launching the tests > The expected behavior is that there are no intermittent failures. Acceptance > criteria: 5 consecutive runs without failures. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (METRON-1340) Improve e2e tests for metron alerts
[ https://issues.apache.org/jira/browse/METRON-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288397#comment-16288397 ] ASF GitHub Bot commented on METRON-1340: Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/857 I was reviewing the protractor documentation [here](http://www.protractortest.org/#/async-await) and I noticed this: > Don’t forget to turn off control_flow, you cannot use a mix of async/await and the control flow: async/await causes the control flow to become unreliable (see github issue). So if you async/await anywhere in a spec, you should use the SELENIUM_PROMISE_MANAGER: false I don't see that setting in protractor.conf.js. Shouldn't it be included? Have all the tests been updated to use async/await instead of web driver flow control? > Improve e2e tests for metron alerts > --- > > Key: METRON-1340 > URL: https://issues.apache.org/jira/browse/METRON-1340 > Project: Metron > Issue Type: Bug >Reporter: RaghuMitra >Assignee: RaghuMitra > > Need to improve e2e tests in the following areas: > - Tests should not be flaky > - Remove the sleep ( This should implicitly make the tests run faster) > - Truncate HBase table 'metron_update' before starting the tests > - Improve the tests descriptions > - Run the tests headless if possible > - Check the node version and browser version before launching the tests > The expected behavior is that there are no intermittent failures. Acceptance > criteria: 5 consecutive runs without failures. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (METRON-1340) Improve e2e tests for metron alerts
[ https://issues.apache.org/jira/browse/METRON-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288272#comment-16288272 ] ASF GitHub Bot commented on METRON-1340: Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/857 OK, I was able to get the tests to run. First run I get a single error. @merrimanr and I are talking through it now. ``` [INFO] Test spec for login page [INFO] ✗ should display error message for invalid credentials [INFO] - Failed: Error while waiting for Protractor to sync with the page: true [INFO] .✓ should login for valid credentials [INFO] .✓ should logout ... [INFO] Failures: [INFO] 1) Test spec for login page should display error message for invalid credentials [INFO] Message: [INFO] Failed: Error while waiting for Protractor to sync with the page: true [INFO] Stack: [INFO] Error: Error while waiting for Protractor to sync with the page: true [INFO] at runWaitForAngularScript.then (/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/built/browser.js:463:23) [INFO] at ManagedPromise.invokeCallback_ (/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1376:14) [INFO] at TaskQueue.execute_ (/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:3084:14) [INFO] at TaskQueue.executeNext_ (/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:3067:27) [INFO] at asyncRun (/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2927:27) [INFO] at /Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:668:7 [INFO] at process._tickCallback (internal/process/next_tick.js:109:7) [INFO] From: Task: [INFO] at pollCondition (/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2195:19) [INFO] at /Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2191:7 [INFO] at new ManagedPromise (/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1077:7) [INFO] at ControlFlow.promise (/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2505:12) [INFO] at /Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2190:22 [INFO] at TaskQueue.execute_ (/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:3084:14) [INFO] at TaskQueue.executeNext_ (/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:3067:27) [INFO] at asyncRun (/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2974:25) [INFO] at /Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:668:7 [INFO] at process._tickCallback (internal/process/next_tick.js:109:7) [INFO] From: Task: [INFO] at scheduleWait (/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2188:20) [INFO] at ControlFlow.wait (/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2517:12) [INFO] at Driver.wait (/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/webdriver.js:934:29) [INFO] at run (/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/built/browser.js:59:33) [INFO] at ProtractorBrowser.to.(anonymous function) [as wait] (/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/built/browser.js:67:16) [INFO] at
[jira] [Commented] (METRON-1340) Improve e2e tests for metron alerts
[ https://issues.apache.org/jira/browse/METRON-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288255#comment-16288255 ] ASF GitHub Bot commented on METRON-1340: Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/857 Should we also run the tests using the new Maven profile? `mvn test -Pe2e` The README should probably be updated to reflect this as well. > Improve e2e tests for metron alerts > --- > > Key: METRON-1340 > URL: https://issues.apache.org/jira/browse/METRON-1340 > Project: Metron > Issue Type: Bug >Reporter: RaghuMitra >Assignee: RaghuMitra > > Need to improve e2e tests in the following areas: > - Tests should not be flaky > - Remove the sleep ( This should implicitly make the tests run faster) > - Truncate HBase table 'metron_update' before starting the tests > - Improve the tests descriptions > - Run the tests headless if possible > - Check the node version and browser version before launching the tests > The expected behavior is that there are no intermittent failures. Acceptance > criteria: 5 consecutive runs without failures. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (METRON-1340) Improve e2e tests for metron alerts
[ https://issues.apache.org/jira/browse/METRON-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288246#comment-16288246 ] ASF GitHub Bot commented on METRON-1340: Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/857 A couple notes for the community - if your node/npm versions are out of date and you've already built the project, you'll run into trouble with versions not matching against the expected node version, e.g. `Node Sass could not find a binding for your current environment: OS X 64-bit with Node.js 9.x`. I deleted the node_modules directory entirely, `rm -r metron-alerts/node_modules`, re-ran the install, `npm install`, and then the start server script ran successfully. Just to note, the server script does not background the process, so you'll need to manually do that or use 2 windows to run the e2e tests. > Improve e2e tests for metron alerts > --- > > Key: METRON-1340 > URL: https://issues.apache.org/jira/browse/METRON-1340 > Project: Metron > Issue Type: Bug >Reporter: RaghuMitra >Assignee: RaghuMitra > > Need to improve e2e tests in the following areas: > - Tests should not be flaky > - Remove the sleep ( This should implicitly make the tests run faster) > - Truncate HBase table 'metron_update' before starting the tests > - Improve the tests descriptions > - Run the tests headless if possible > - Check the node version and browser version before launching the tests > The expected behavior is that there are no intermittent failures. Acceptance > criteria: 5 consecutive runs without failures. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (METRON-1340) Improve e2e tests for metron alerts
[ https://issues.apache.org/jira/browse/METRON-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288208#comment-16288208 ] ASF GitHub Bot commented on METRON-1340: Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/857 Seems that my NodeJS and npm versions got mangled since last time I ran this. Perhaps migrating laptops, upgrading to High Sierra, and switching from MacPorts to HomeBrew might have something to do with it ;) > Improve e2e tests for metron alerts > --- > > Key: METRON-1340 > URL: https://issues.apache.org/jira/browse/METRON-1340 > Project: Metron > Issue Type: Bug >Reporter: RaghuMitra >Assignee: RaghuMitra > > Need to improve e2e tests in the following areas: > - Tests should not be flaky > - Remove the sleep ( This should implicitly make the tests run faster) > - Truncate HBase table 'metron_update' before starting the tests > - Improve the tests descriptions > - Run the tests headless if possible > - Check the node version and browser version before launching the tests > The expected behavior is that there are no intermittent failures. Acceptance > criteria: 5 consecutive runs without failures. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (METRON-1340) Improve e2e tests for metron alerts
[
https://issues.apache.org/jira/browse/METRON-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288176#comment-16288176
]
ASF GitHub Bot commented on METRON-1340:
Github user mmiklavc commented on the issue:
https://github.com/apache/metron/pull/857
Something I'm missing?
```
./scripts/start-server-for-e2e.sh
/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/@angular/cli/bin/ng:7
const CliConfig = require('../models/config').CliConfig;
^
SyntaxError: Use of const in strict mode.
at Module._compile (module.js:439:25)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Function.Module.runMain (module.js:497:10)
at startup (node.js:119:16)
at node.js:945:3
```
> Improve e2e tests for metron alerts
> ---
>
> Key: METRON-1340
> URL: https://issues.apache.org/jira/browse/METRON-1340
> Project: Metron
> Issue Type: Bug
>Reporter: RaghuMitra
>Assignee: RaghuMitra
>
> Need to improve e2e tests in the following areas:
> - Tests should not be flaky
> - Remove the sleep ( This should implicitly make the tests run faster)
> - Truncate HBase table 'metron_update' before starting the tests
> - Improve the tests descriptions
> - Run the tests headless if possible
> - Check the node version and browser version before launching the tests
> The expected behavior is that there are no intermittent failures. Acceptance
> criteria: 5 consecutive runs without failures.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (METRON-1340) Improve e2e tests for metron alerts
[ https://issues.apache.org/jira/browse/METRON-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286209#comment-16286209 ] ASF GitHub Bot commented on METRON-1340: Github user justinleet commented on the issue: https://github.com/apache/metron/pull/857 I've hit the same intermittent errors trying this, e.g. ``` [INFO] - Failed: unknown error: Element ... is not clickable at point (1279, 95). Other element would receive the click: ... ``` One of the very first things that failed was ``` [INFO] ✗ should display error message for invalid credentials [INFO] - Failed: Error while waiting for Protractor to sync with the page: true ``` Is it possible there's an intermittent error getting the tests initially lined up that just carries through everything? @iraghumitra Can you also update the README with the instructions for running this with the mvn profile + any troubleshooting or other steps that need to happen? > Improve e2e tests for metron alerts > --- > > Key: METRON-1340 > URL: https://issues.apache.org/jira/browse/METRON-1340 > Project: Metron > Issue Type: Bug >Reporter: RaghuMitra >Assignee: RaghuMitra > > Need to improve e2e tests in the following areas: > - Tests should not be flaky > - Remove the sleep ( This should implicitly make the tests run faster) > - Truncate HBase table 'metron_update' before starting the tests > - Improve the tests descriptions > - Run the tests headless if possible > - Check the node version and browser version before launching the tests > The expected behavior is that there are no intermittent failures. Acceptance > criteria: 5 consecutive runs without failures. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (METRON-1340) Improve e2e tests for metron alerts
[ https://issues.apache.org/jira/browse/METRON-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280606#comment-16280606 ] ASF GitHub Bot commented on METRON-1340: Github user iraghumitra commented on the issue: https://github.com/apache/metron/pull/857 @merrimanr renamed the variable POLLING_DEFAULT_STATE to DISABLE_POLLING > Improve e2e tests for metron alerts > --- > > Key: METRON-1340 > URL: https://issues.apache.org/jira/browse/METRON-1340 > Project: Metron > Issue Type: Bug >Reporter: RaghuMitra >Assignee: RaghuMitra > > Need to improve e2e tests in the following areas: > - Tests should not be flaky > - Remove the sleep ( This should implicitly make the tests run faster) > - Truncate HBase table 'metron_update' before starting the tests > - Improve the tests descriptions > - Run the tests headless if possible > - Check the node version and browser version before launching the tests > The expected behavior is that there are no intermittent failures. Acceptance > criteria: 5 consecutive runs without failures. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (METRON-1340) Improve e2e tests for metron alerts
[
https://issues.apache.org/jira/browse/METRON-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280278#comment-16280278
]
ASF GitHub Bot commented on METRON-1340:
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/metron/pull/857#discussion_r155261159
--- Diff: metron-interface/metron-alerts/src/app/utils/constants.ts ---
@@ -38,5 +38,6 @@ export let TREE_SUB_GROUP_SIZE = 5;
export let DEFAULT_FACETS = ['source:type', 'ip_src_addr', 'ip_dst_addr',
'host', 'enrichments:geo:ip_dst_addr:country'];
export let DEFAULT_GROUPS = ['source:type', 'ip_src_addr', 'ip_dst_addr',
'host', 'enrichments:geo:ip_dst_addr:country'];
export let INDEXES = environment.indices ? environment.indices.split(',')
: [];
+export let POLLING_DEFAULT_STATE = environment.defaultPollingState;
--- End diff --
Would it be possible to change this to a better name? Something like
DISABLE_POLLING? Setting POLLING_DEFAULT_STATE to true to stop polling doesn't
make sense to me.
> Improve e2e tests for metron alerts
> ---
>
> Key: METRON-1340
> URL: https://issues.apache.org/jira/browse/METRON-1340
> Project: Metron
> Issue Type: Bug
>Reporter: RaghuMitra
>Assignee: RaghuMitra
>
> Need to improve e2e tests in the following areas:
> - Tests should not be flaky
> - Remove the sleep ( This should implicitly make the tests run faster)
> - Truncate HBase table 'metron_update' before starting the tests
> - Improve the tests descriptions
> - Run the tests headless if possible
> - Check the node version and browser version before launching the tests
> The expected behavior is that there are no intermittent failures. Acceptance
> criteria: 5 consecutive runs without failures.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (METRON-1340) Improve e2e tests for metron alerts
[
https://issues.apache.org/jira/browse/METRON-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280096#comment-16280096
]
ASF GitHub Bot commented on METRON-1340:
Github user iraghumitra commented on a diff in the pull request:
https://github.com/apache/metron/pull/857#discussion_r155219994
--- Diff: metron-interface/metron-alerts/src/app/utils/constants.ts ---
@@ -38,5 +38,6 @@ export let TREE_SUB_GROUP_SIZE = 5;
export let DEFAULT_FACETS = ['source:type', 'ip_src_addr', 'ip_dst_addr',
'host', 'enrichments:geo:ip_dst_addr:country'];
export let DEFAULT_GROUPS = ['source:type', 'ip_src_addr', 'ip_dst_addr',
'host', 'enrichments:geo:ip_dst_addr:country'];
export let INDEXES = environment.indices ? environment.indices.split(',')
: [];
+export let POLLING_DEFAULT_STATE = environment.defaultPollingState;
--- End diff --
Added info for this in Readme and CC
> Improve e2e tests for metron alerts
> ---
>
> Key: METRON-1340
> URL: https://issues.apache.org/jira/browse/METRON-1340
> Project: Metron
> Issue Type: Bug
>Reporter: RaghuMitra
>Assignee: RaghuMitra
>
> Need to improve e2e tests in the following areas:
> - Tests should not be flaky
> - Remove the sleep ( This should implicitly make the tests run faster)
> - Truncate HBase table 'metron_update' before starting the tests
> - Improve the tests descriptions
> - Run the tests headless if possible
> - Check the node version and browser version before launching the tests
> The expected behavior is that there are no intermittent failures. Acceptance
> criteria: 5 consecutive runs without failures.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (METRON-1340) Improve e2e tests for metron alerts
[ https://issues.apache.org/jira/browse/METRON-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280091#comment-16280091 ] ASF GitHub Bot commented on METRON-1340: Github user iraghumitra commented on the issue: https://github.com/apache/metron/pull/857 @merrimanr - I removed the node version check in the ui code - The browser.sleep is used in following areas - When we are using methods like waitForText, few times I see the test fails saying that that it got the text we were waiting to change which doesn't make sense. So, in this case, i am just yielding the main thread with a sleep of 500 milliseconds. - Create/Add meta-alert, these calls takes few seconds at times. - When we use animation to slide in the details pane, we don't have a class that gets added/removed for this so it makes it difficult for us to know when the pane is loaded. We can add/remove a class for animation by moving the slider to a component or directive. This might be out of scope for current PR. - Refactored tests for Facets - Looks like the meta alert creation failed can you attach the stack trace it might give me additional info to debug. > Improve e2e tests for metron alerts > --- > > Key: METRON-1340 > URL: https://issues.apache.org/jira/browse/METRON-1340 > Project: Metron > Issue Type: Bug >Reporter: RaghuMitra >Assignee: RaghuMitra > > Need to improve e2e tests in the following areas: > - Tests should not be flaky > - Remove the sleep ( This should implicitly make the tests run faster) > - Truncate HBase table 'metron_update' before starting the tests > - Improve the tests descriptions > - Run the tests headless if possible > - Check the node version and browser version before launching the tests > The expected behavior is that there are no intermittent failures. Acceptance > criteria: 5 consecutive runs without failures. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (METRON-1340) Improve e2e tests for metron alerts
[ https://issues.apache.org/jira/browse/METRON-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279170#comment-16279170 ] ASF GitHub Bot commented on METRON-1340: Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/857 Took a first pass at this and I feel like the e2e tests are much improved. Great progress and good job so far. I was able to get several successful runs whereas before it was difficult to get a single successful run. A couple of comments/suggestions: - I noticed a warning message stating I wasn't on a recent enough version of nodejs. I think we should be using the maven frontend plugin to run tests so that we guarantee a consistent version is used. I submitted a [PR](https://github.com/iraghumitra/incubator-metron/pull/6) as an example of how to do this. - I noticed there are still several browser.sleep statements throughout the tests (I counted 20). I think our goal should be to remove all of them. I know some of these should definitely be removed (alerts-list.po.ts, line 87) and may have just been missed. If there are cases where we MUST have them, I think those cases needed to be discussed and justified. - I feel like the "should expand all facets" and "should collapse all facets" tests in alert-filters.e2e-spec.ts are unnecessary (and would allow us to remove a couple browser.sleep statements). These tests are simply opening and closing bootstrap widgets which is controlled by code we don't maintain (bootstrap). I would instead prefer a test that selects a filter and checks that the search box and results are properly updated. - I have ran into these errors a couple times. Not sure if I just ended up in a bad state somehow or if it's because I'm running tests through Maven: ``` ✗ should have all the steps for meta alerts workflow - Failed: unknown error: Element is not clickable at point (1278, 102). Other element would receive the click: ... ✗ should create a meta alert from nesting of more than one level - Failed: unknown error: Element is not clickable at point (1350, 503). Other element would receive the click: ... ``` > Improve e2e tests for metron alerts > --- > > Key: METRON-1340 > URL: https://issues.apache.org/jira/browse/METRON-1340 > Project: Metron > Issue Type: Bug >Reporter: RaghuMitra >Assignee: RaghuMitra > > Need to improve e2e tests in the following areas: > - Tests should not be flaky > - Remove the sleep ( This should implicitly make the tests run faster) > - Truncate HBase table 'metron_update' before starting the tests > - Improve the tests descriptions > - Run the tests headless if possible > - Check the node version and browser version before launching the tests > The expected behavior is that there are no intermittent failures. Acceptance > criteria: 5 consecutive runs without failures. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (METRON-1340) Improve e2e tests for metron alerts
[
https://issues.apache.org/jira/browse/METRON-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279147#comment-16279147
]
ASF GitHub Bot commented on METRON-1340:
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/metron/pull/857#discussion_r155067123
--- Diff: metron-interface/metron-alerts/src/app/utils/constants.ts ---
@@ -38,5 +38,6 @@ export let TREE_SUB_GROUP_SIZE = 5;
export let DEFAULT_FACETS = ['source:type', 'ip_src_addr', 'ip_dst_addr',
'host', 'enrichments:geo:ip_dst_addr:country'];
export let DEFAULT_GROUPS = ['source:type', 'ip_src_addr', 'ip_dst_addr',
'host', 'enrichments:geo:ip_dst_addr:country'];
export let INDEXES = environment.indices ? environment.indices.split(',')
: [];
+export let POLLING_DEFAULT_STATE = environment.defaultPollingState;
--- End diff --
What does the POLLING_DEFAULT_STATE setting do?
> Improve e2e tests for metron alerts
> ---
>
> Key: METRON-1340
> URL: https://issues.apache.org/jira/browse/METRON-1340
> Project: Metron
> Issue Type: Bug
>Reporter: RaghuMitra
>Assignee: RaghuMitra
>
> Need to improve e2e tests in the following areas:
> - Tests should not be flaky
> - Remove the sleep ( This should implicitly make the tests run faster)
> - Truncate HBase table 'metron_update' before starting the tests
> - Improve the tests descriptions
> - Run the tests headless if possible
> - Check the node version and browser version before launching the tests
> The expected behavior is that there are no intermittent failures. Acceptance
> criteria: 5 consecutive runs without failures.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (METRON-1340) Improve e2e tests for metron alerts
[ https://issues.apache.org/jira/browse/METRON-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16276852#comment-16276852 ] ASF GitHub Bot commented on METRON-1340: Github user justinleet commented on the issue: https://github.com/apache/metron/pull/857 I haven't looked at the code yet, but I was able to pull this down and run the e2e tests repeatedly without failures, so this is definitely great stuff. > Improve e2e tests for metron alerts > --- > > Key: METRON-1340 > URL: https://issues.apache.org/jira/browse/METRON-1340 > Project: Metron > Issue Type: Bug >Reporter: RaghuMitra >Assignee: RaghuMitra > > Need to improve e2e tests in the following areas: > - Tests should not be flaky > - Remove the sleep ( This should implicitly make the tests run faster) > - Truncate HBase table 'metron_update' before starting the tests > - Improve the tests descriptions > - Run the tests headless if possible > - Check the node version and browser version before launching the tests > The expected behavior is that there are no intermittent failures. Acceptance > criteria: 5 consecutive runs without failures. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (METRON-1340) Improve e2e tests for metron alerts
[ https://issues.apache.org/jira/browse/METRON-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16276033#comment-16276033 ] ASF GitHub Bot commented on METRON-1340: GitHub user iraghumitra opened a pull request: https://github.com/apache/metron/pull/857 METRON-1340: Improve e2e tests for metron alerts ## Contributor Comments This PR aims to improve the existing e2e tests of metro-alerts in the areas mentioned below. The PR doesn't add/remove any functionality to the existing alerts ui. However, there are few changes in HTML templates where I a needed a new selector for tests. - Delete the contents of table 'metron_update' in HBase - I have added a nodes script that deletes the 'metron_update' table before each run of e2e tests - Ensure that the tests run consistently on latest version of node - I have tested and added a check to node version 8.9. If the version is less than 8.9 there is a warning and tests run continues - Improving description of test specs - Added meaningful descriptions wherever applicable. Let me know if we need any more descriptions - Run the tests on a headless browser to improve reliability - Added the headless option to protractor.conf.js The tests are written using: OS: macOS High Sierra 10.13.1 nodejs: v8.9.1. Chrome: 62.0.3202.94 The tests should run reasonably well on other platforms since I personally didn't test on them I would request you to report any tests failure with information about your OS, nodejs and browser version. I bumped up the version of the protractor to 5.2.0, so please run npm install before testing the PR. Please let me know if you need any more details about the PR. ### Testing If all the e2e tests run fine the PR is good. ### Documentation No new documentation is needed specifically for this PR. The steps to run the e2e tests remains the way they are before. I added a couple of caveats in Readme about deleting 'metron_update' table and headless chrome. Please let me know if we can improve our docs for e2e tests I would be glad to add it. ## Pull Request Checklist Thank you for submitting a contribution to Apache Metron. Please refer to our [Development Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235) for the complete guide to follow for contributions. Please refer also to our [Build Verification Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview) for complete smoke testing guides. In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel). - [x] Does your PR title start with METRON- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? ### For code changes: - [x] Have you included steps to reproduce the behavior or problem that is being changed or addressed? - [x] Have you included steps or a guide to how the change may be verified and tested manually? - [x] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via: ``` mvn -q clean integration-test install && build_utils/verify_licenses.sh ``` - [x] Have you written or updated unit tests and or integration tests to verify your changes? - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`: ``` cd site-book mvn site ``` Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. It is also recommended that [travis-ci](https://travis-ci.org) is set up for your personal repository such that your branches are built there before submitting a
