[jira] [Commented] (RANGER-2255) Add JavaScript unit tests
[ https://issues.apache.org/jira/browse/RANGER-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1672#comment-1672 ] Csaba Koncz commented on RANGER-2255: - RANGER-2321 was merged. However, it does not exactly act as outlined above: the default is to execute JavaScript tests (just like Java tests are run by default). -DskipJSTests is needed to skip JavaScript tests (including PhantomJS downloading) -DskipTests skips both Java and JS tests. > Add JavaScript unit tests > - > > Key: RANGER-2255 > URL: https://issues.apache.org/jira/browse/RANGER-2255 > Project: Ranger > Issue Type: Wish > Components: admin >Affects Versions: 0.7.0, 2.0.0, 1.2.1 >Reporter: Csaba Koncz >Assignee: Csaba Koncz >Priority: Minor > Fix For: 2.0.0 > > > It would be nice if the admin-ui project would have JavaScript unit tests. > As with RANGER-2220 JavaScript minification was introduced, that can lead to > new type of loading errors that were not seen before. > It would be nice if there was an automatic check that validates the minified > output. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (RANGER-2255) Add JavaScript unit tests
[ https://issues.apache.org/jira/browse/RANGER-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740264#comment-16740264 ] Pradeep Agrawal commented on RANGER-2255: - [~Koncz] It will be okay to make JS Tests run optional. > Add JavaScript unit tests > - > > Key: RANGER-2255 > URL: https://issues.apache.org/jira/browse/RANGER-2255 > Project: Ranger > Issue Type: Wish > Components: admin >Affects Versions: 0.7.0, 2.0.0, 1.2.1 >Reporter: Csaba Koncz >Assignee: Csaba Koncz >Priority: Minor > Fix For: 2.0.0 > > > It would be nice if the admin-ui project would have JavaScript unit tests. > As with RANGER-2220 JavaScript minification was introduced, that can lead to > new type of loading errors that were not seen before. > It would be nice if there was an automatic check that validates the minified > output. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (RANGER-2255) Add JavaScript unit tests
[ https://issues.apache.org/jira/browse/RANGER-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739227#comment-16739227 ] Csaba Koncz commented on RANGER-2255: - Reverted the patch belonging to RANGER-2319 to its original version (which only removes the deprecated packege). Created a separated issue for the build failure: RANGER-2321 > Add JavaScript unit tests > - > > Key: RANGER-2255 > URL: https://issues.apache.org/jira/browse/RANGER-2255 > Project: Ranger > Issue Type: Wish > Components: admin >Affects Versions: 0.7.0, 2.0.0, 1.2.1 >Reporter: Csaba Koncz >Assignee: Csaba Koncz >Priority: Minor > Fix For: 2.0.0 > > > It would be nice if the admin-ui project would have JavaScript unit tests. > As with RANGER-2220 JavaScript minification was introduced, that can lead to > new type of loading errors that were not seen before. > It would be nice if there was an automatic check that validates the minified > output. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (RANGER-2255) Add JavaScript unit tests
[
https://issues.apache.org/jira/browse/RANGER-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738293#comment-16738293
]
Csaba Koncz commented on RANGER-2255:
-
[~pradeep.agrawal]: updated the review at [https://reviews.apache.org/r/69693/]
so that JavaScript testing is not done unless explicitly requested.
I.e.
{code:java}
mvn clean verify -am pl security-admin{code}
will run only Java tests and will not try to install Karma, Mocha and PhantomJS.
The command to run JS tests is:
{code:java}
mvn clean verify -am pl security-admin -DskipJSTests=false{code}
(we might want to add this to the Travis config)
Is this acceptable?
> Add JavaScript unit tests
> -
>
> Key: RANGER-2255
> URL: https://issues.apache.org/jira/browse/RANGER-2255
> Project: Ranger
> Issue Type: Wish
> Components: admin
>Affects Versions: 0.7.0, 2.0.0, 1.2.1
>Reporter: Csaba Koncz
>Assignee: Csaba Koncz
>Priority: Minor
> Fix For: 2.0.0
>
>
> It would be nice if the admin-ui project would have JavaScript unit tests.
> As with RANGER-2220 JavaScript minification was introduced, that can lead to
> new type of loading errors that were not seen before.
> It would be nice if there was an automatic check that validates the minified
> output.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (RANGER-2255) Add JavaScript unit tests
[ https://issues.apache.org/jira/browse/RANGER-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738135#comment-16738135 ] Csaba Koncz commented on RANGER-2255: - [~pradeep.agrawal]: my change does not add any functionality, so yes, it can be reverted. On the other hand I would like to have some minimal JavaScript testing, so can we try to save it? I ran npm in a clean CentOS container (centos:7.5.1804) and it failed for me, too. It turned out that I need to install bzip2 and fontconfig. (yum install bzip2 fontconfig). Fontconfig is not needed for installation, only for the PhantomJS binary itself. The situation is the same with Debian 9: `apt-get update; apt-get install bzip2 fontconfig` is needed. Additionally, I can make JavaScript test execution disabled by default and run it only when an extra system property is specified (.e.g. -Djstests=true). > Add JavaScript unit tests > - > > Key: RANGER-2255 > URL: https://issues.apache.org/jira/browse/RANGER-2255 > Project: Ranger > Issue Type: Wish > Components: admin >Affects Versions: 0.7.0, 2.0.0, 1.2.1 >Reporter: Csaba Koncz >Assignee: Csaba Koncz >Priority: Minor > Fix For: 2.0.0 > > > It would be nice if the admin-ui project would have JavaScript unit tests. > As with RANGER-2220 JavaScript minification was introduced, that can lead to > new type of loading errors that were not seen before. > It would be nice if there was an automatic check that validates the minified > output. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (RANGER-2255) Add JavaScript unit tests
[ https://issues.apache.org/jira/browse/RANGER-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737854#comment-16737854 ] Pradeep Agrawal commented on RANGER-2255: - [~Koncz] : Can we revert the patch if there are no solutions for the Linux environment. I am using centos 7 env. > Add JavaScript unit tests > - > > Key: RANGER-2255 > URL: https://issues.apache.org/jira/browse/RANGER-2255 > Project: Ranger > Issue Type: Wish > Components: admin >Affects Versions: 0.7.0, 2.0.0, 1.2.1 >Reporter: Csaba Koncz >Assignee: Csaba Koncz >Priority: Minor > Fix For: 2.0.0 > > > It would be nice if the admin-ui project would have JavaScript unit tests. > As with RANGER-2220 JavaScript minification was introduced, that can lead to > new type of loading errors that were not seen before. > It would be nice if there was an automatic check that validates the minified > output. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (RANGER-2255) Add JavaScript unit tests
[
https://issues.apache.org/jira/browse/RANGER-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737025#comment-16737025
]
Csaba Koncz commented on RANGER-2255:
-
[~pradeep.agrawal], I noticed that two Phantomjs versions are currently
downloaded by npm, one of which is deprecated.
Removed the deprecated one (RANGER-2319), but this probably will not fix your
issue, as on Linux it is still a BZIP archive that is downloaded.
I tested the installation with Docker image node:8.15.0-alpine, there it works:
{code:java}
# npm i
> phantomjs-prebuilt@2.1.16 install /root/csaba/node_modules/phantomjs-prebuilt
> node install.js
PhantomJS not found on PATH
Downloading
https://github.com/Medium/phantomjs/releases/download/v2.1.1/phantomjs-2.1.1-linux-x86_64.tar.bz2
Saving to /tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2
Receiving...
[] 100%
Received 22866K total.
Extracting tar contents (via spawned process)
Removing /root/csaba/node_modules/phantomjs-prebuilt/lib/phantom
Copying extracted folder
/tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2-extract-1546944498271/phantomjs-2.1.1-linux-x86_64
-> /root/csaba/node_modules/phantomjs-prebuilt/lib/phantom
Writing location.js file
Done. Phantomjs binary available at
/root/csaba/node_modules/phantomjs-prebuilt/lib/phantom/bin/phantomjs
npm WARN security-admin@2.0.0 No description
npm WARN security-admin@2.0.0 No repository field.
npm WARN security-admin@2.0.0 No license field.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.4
(node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for
fsevents@1.2.4: wanted {"os":"darwin","arch":"any"} (current:
{"os":"linux","arch":"x64"})
added 349 packages from 1057 contributors and audited 2496 packages in 48.538s
found 0 vulnerabilities{code}
> Add JavaScript unit tests
> -
>
> Key: RANGER-2255
> URL: https://issues.apache.org/jira/browse/RANGER-2255
> Project: Ranger
> Issue Type: Wish
> Components: admin
>Affects Versions: 0.7.0, 2.0.0, 1.2.1
>Reporter: Csaba Koncz
>Assignee: Csaba Koncz
>Priority: Minor
> Fix For: 2.0.0
>
>
> It would be nice if the admin-ui project would have JavaScript unit tests.
> As with RANGER-2220 JavaScript minification was introduced, that can lead to
> new type of loading errors that were not seen before.
> It would be nice if there was an automatic check that validates the minified
> output.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (RANGER-2255) Add JavaScript unit tests
[ https://issues.apache.org/jira/browse/RANGER-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736938#comment-16736938 ] Csaba Koncz commented on RANGER-2255: - Sorry [~pradeep.agrawal], I was not aware of the dependency on bzip2. Others seem to have seen the same issue: [https://github.com/Medium/phantomjs/issues/92#issuecomment-22908035] Is it an option for you to install bzip2 to your machine? Unfortunately I have no other guesses right now (do not know if I can get npm download some other type of archive). Best regards, Csaba > Add JavaScript unit tests > - > > Key: RANGER-2255 > URL: https://issues.apache.org/jira/browse/RANGER-2255 > Project: Ranger > Issue Type: Wish > Components: admin >Affects Versions: 0.7.0, 2.0.0, 1.2.1 >Reporter: Csaba Koncz >Assignee: Csaba Koncz >Priority: Minor > Fix For: 2.0.0 > > > It would be nice if the admin-ui project would have JavaScript unit tests. > As with RANGER-2220 JavaScript minification was introduced, that can lead to > new type of loading errors that were not seen before. > It would be nice if there was an automatic check that validates the minified > output. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (RANGER-2255) Add JavaScript unit tests
[
https://issues.apache.org/jira/browse/RANGER-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736884#comment-16736884
]
Pradeep Agrawal commented on RANGER-2255:
-
[~Koncz]/Zsombor Gegesy: Build is failing for me with the following error:
{code:java}
[INFO] Extracting tar contents (via spawned process)
[ERROR] Error extracting archive
[ERROR] Phantom installation failed { Error: Command failed: tar jxf
/tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2
[ERROR] tar (child): bzip2: Cannot exec: No such file or directory
[ERROR] tar (child): Error is not recoverable: exiting now
[ERROR] tar: Child returned status 2
[ERROR] tar: Error is not recoverable: exiting now
[ERROR]
[ERROR] at ChildProcess.exithandler (child_process.js:276:12)
[ERROR] at emitTwo (events.js:126:13)
[ERROR] at ChildProcess.emit (events.js:214:7)
[ERROR] at maybeClose (internal/child_process.js:915:16)
[ERROR] at Socket.stream.socket.on (internal/child_process.js:336:11)
[ERROR] at emitOne (events.js:116:13)
[ERROR] at Socket.emit (events.js:211:7)
[ERROR] at Pipe._handle.close [as _onclose] (net.js:561:12)
[ERROR] killed: false,
[ERROR] code: 2,
[ERROR] signal: null,
[ERROR] cmd: 'tar jxf /tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2' }
Error: Command failed: tar jxf
/tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2
[ERROR] tar (child): bzip2: Cannot exec: No such file or directory
[ERROR] tar (child): Error is not recoverable: exiting now
[ERROR] tar: Child returned status 2
[ERROR] tar: Error is not recoverable: exiting now
[ERROR]
[ERROR] at ChildProcess.exithandler (child_process.js:276:12)
[ERROR] at emitTwo (events.js:126:13)
[ERROR] at ChildProcess.emit (events.js:214:7)
[ERROR] at maybeClose (internal/child_process.js:915:16)
[ERROR] at Socket.stream.socket.on (internal/child_process.js:336:11)
[ERROR] at emitOne (events.js:116:13)
[ERROR] at Socket.emit (events.js:211:7)
[ERROR] at Pipe._handle.close [as _onclose] (net.js:561:12)
[WARNING] npm WARN security-admin@2.0.0 No description
[WARNING] npm WARN security-admin@2.0.0 No repository field.
[WARNING] npm WARN security-admin@2.0.0 No license field.
[WARNING] npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.4
(node_modules/fsevents):
[WARNING] npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform
for fsevents@1.2.4: wanted {"os":"darwin","arch":"any"} (current:
{"os":"linux","arch":"x64"})
[ERROR]
[ERROR] npm ERR! code ELIFECYCLE
[ERROR] npm ERR! errno 1
[ERROR] npm ERR! phantomjs-prebuilt@2.1.16 install: `node install.js`
[ERROR] npm ERR! Exit status 1
[ERROR] npm ERR!
[ERROR] npm ERR! Failed at the phantomjs-prebuilt@2.1.16 install script.
[ERROR] npm ERR! This is probably not a problem with npm. There is likely
additional logging output above.
[ERROR]
[ERROR] npm ERR! A complete log of this run can be found in:
[ERROR] npm ERR! /root/.npm/_logs/2019-01-08T08_37_19_072Z-debug.log
[INFO]
[INFO]
[INFO] Skipping ranger
[INFO] This project has been banned from the build due to previous failures.
[INFO]
[INFO]
[INFO]
[INFO] Skipping ranger
[INFO] This project has been banned from the build due to previous failures.
[INFO]
[INFO]
[INFO] Reactor Summary:
[INFO]
[INFO] ranger . SUCCESS [ 2.037 s]
[INFO] Jdbc SQL Connector . SUCCESS [ 1.962 s]
[INFO] Credential Support . SUCCESS [ 3.551 s]
[INFO] Audit Component SUCCESS [ 4.429 s]
[INFO] Common library for Plugins . SUCCESS [ 8.583 s]
[INFO] Installer Support Component SUCCESS [ 1.103 s]
[INFO] Credential Builder . SUCCESS [ 2.857 s]
[INFO] Embedded Web Server Invoker SUCCESS [ 2.144 s]
[INFO] Key Management Service . SUCCESS [ 5.090 s]
[INFO] ranger-plugin-classloader .. SUCCESS [ 1.642 s]
[INFO] HBase Security Plugin Shim . SUCCESS [ 9.623 s]
[INFO] HBase Security Plugin .. SUCCESS [ 6.574 s]
[INFO] Hdfs Security Plugin ... SUCCESS [ 6.651 s]
[INFO] Hive Security Plugin ... SUCCESS [ 12.996 s]
[INFO] Knox Security Plugin Shim .. SUCCESS [ 3.095 s]
[INFO] Knox
[jira] [Commented] (RANGER-2255) Add JavaScript unit tests
[ https://issues.apache.org/jira/browse/RANGER-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16654137#comment-16654137 ] Csaba Koncz commented on RANGER-2255: - Added JavaScript testing infrastructure (Karma, Mocha, PhantomJS) and a minimal test that checks whether the JavaScript application can be (hopefully) loaded: [https://reviews.apache.org/r/69067/] This required two changes in the production code: # Main.js was converted to an AMD module (it did not have a define wrapper, now it has) # Main.js loading is triggered by a script tag in index.html. The production and development versions now differ only in how the JavaScript assets are loaded into the browser: ## by letting RequireJS know the asset locations in Init.js and then rely on the recursive download ## by inlining all resources via Main.min.js. No RequireJS configuration is needed in this case. The test execution does not use index.html but has two configurations that differ only in the JS module loading method. > Add JavaScript unit tests > - > > Key: RANGER-2255 > URL: https://issues.apache.org/jira/browse/RANGER-2255 > Project: Ranger > Issue Type: Wish > Components: admin >Affects Versions: 0.7.0, 2.0.0, 1.2.1 >Reporter: Csaba Koncz >Priority: Minor > Fix For: 2.0.0 > > > It would be nice if the admin-ui project would have JavaScript unit tests. > As with RANGER-2220 JavaScript minification was introduced, that can lead to > new type of loading errors that were not seen before. > It would be nice if there was an automatic check that validates the minified > output. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
