date:20180202

Re: Review Request 65484: RANGER-1969 - Fix failing Kafka tests with latest maven/JVM

2018-02-02 Thread Ramesh Mani


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65484/#review196761
---


Ship it!




Ship It!

- Ramesh Mani


On Feb. 2, 2018, 2:56 p.m., Colm O hEigeartaigh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65484/
> ---
> 
> (Updated Feb. 2, 2018, 2:56 p.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-1969
> https://issues.apache.org/jira/browse/RANGER-1969
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> The Kafka SASL/SSL test is crashing for me with the latest Maven/JVM. 
> Upgrading the surefire plugin solves the problem.
> 
> 
> Diffs
> -
> 
>   pom.xml 255b02aa 
> 
> 
> Diff: https://reviews.apache.org/r/65484/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Colm O hEigeartaigh
> 
>

Re: servicedef validation - implClass

2018-02-02 Thread Ramesh Mani

Nigel,

Could you please provide the exception stack that is printed along with
error message you are showing. Please check in xa_portal.log ( Ranger log).

Also check in Ranger class path ranger-plugins-common*jar is there. Check
in {install.dir}/ranger-admin/ews/lib and
{install.dir}/ranger-admin/ews/webapp/WEB-INF/lib and it has the necessary
permission for the process user who start ranger admin.

As you notice it should have used the base implementation when you leave
implClass blank / or not in the service def.

Regards,
Ramesh


On 2/2/18, 6:24 AM, "Nigel Jones"  wrote:

>We're currently writing a Ranger plugin for an open source virtual
>database driver known as ŒGaianDB¹.
>
>We have the basics of a plugin working, but today a colleague tried to
>setup our plugin, and on their ranger install they could not create an
>instance of our ranger service, hitting the error
>
>"Gaian failed to find service class
>org.apache.ranger.services.gaiandb.RangerServiceGaian. Resource lookup
>will not be available. Please make sure plugin jar is in the correct
>place.²
>
>I had expected this to be a warning (fair enough), but it actually
>prevents the UI from saving the service instance, and so Gaian related
>policies cannot be created.
>
>They got this on I think a) an old 0.6.x build, and 0.7.1.
>Meanwhile I had been using a HDP 2.6.3 setup, which comes with 0.7.0 and
>with the same service definition did NOT of course hit this error.
>
>We haven¹t actually deployed any plugin code to the ranger server (yet)
>since we have not implemented the resource lookup capability. We had
>though specified an implClass in the servicedef. It¹s a later task :-)
>
>Looking at the ranger code in ServiceMgr.java it seems as if an empty
>string should cause ranger to use a default class. We tried this, as well
>as specifying the org.apache.ranger.plugin.service.RangerDefaultService
>class, but had the same error in both cases. I was though looking at code
>in master and see there have been a few changes
>
>Any suggestions as to how to persuade 0.7.0 to load the servicedef (other
>than implement the class required, albeit a no-op oneŠ?), or an
>explanation as to why we might see different behaviour?
>
>I¹m setting up a ranger install off master to debug/check latest codeŠ.
>
>For info the servicedef begins:
>{
>  "id":99,
>  "name": "gaian",
>  "implClass": "org.apache.ranger.services.gaiandb.RangerServiceGaian",
>  "label": "Gaian",
>  "description": "Gaian",
>  "options": {
>"enableDenyAndExceptionsInPolicies": "true"
>  },
>"guid": "86d10748-e4fc-442b-8991-f6a727054ece",
>
>  "resources": [
>The full version can be found in
>https://issues.apache.org/jira/browse/RANGER-1699
> - see the latest
>attachment. Not as a patch as currently getting things working in a diff.
>Build tree
>
>If it seems like a bug (rather than a newbie error or specific to hdp)
>let me know and I¹ll open a jira!
>
>Many thanks
>Nigel.
>
>
>

Re: Review Request 65485: RANGER-1971 - Switch to use for-each loops

2018-02-02 Thread Zsombor Gegesy


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65485/#review196748
---


Ship it!




I think, this can be merged as is.


agents-common/src/test/java/org/apache/ranger/plugin/policyengine/TestCacheMap.java
Line 110 (original), 109 (patched)


If this wasn't a test class, I would suggest to switch the if with the for.



ugsync/src/main/java/org/apache/ranger/unixusersync/process/PolicyMgrUserGroupBuilder.java
Line 431 (original), 430 (patched)


Instead of 
if (map.containsKey(x)) {
   ...   map.get(x)
}

value = map.get(x);
if (value != null) { 
   ... value
}


- Zsombor Gegesy


On febr. 2, 2018, 3:58 du, Colm O hEigeartaigh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65485/
> ---
> 
> (Updated febr. 2, 2018, 3:58 du)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-1971
> https://issues.apache.org/jira/browse/RANGER-1971
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> We should switch to use for-each loops in the code, as it is more readable 
> and concise than using an index.
> 
> 
> Diffs
> -
> 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/provider/hdfs/HdfsLogDestination.java
>  065e8b07 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/queue/AuditFileSpool.java 
> 9abd99f5 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/geo/GeolocationMetadata.java
>  d27a0308 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/geo/RangerGeolocationData.java
>  99d6027f 
>   
> agents-common/src/test/java/org/apache/ranger/plugin/policyengine/TestCacheMap.java
>  5f39b224 
>   
> credentialbuilder/src/main/java/org/apache/ranger/credentialapi/buildks.java 
> b8bdb6fa 
>   
> hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/RangerAuthorizationCoprocessor.java
>  e30f7957 
>   
> hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java
>  fa84b138 
>   jisql/src/main/java/org/apache/util/sql/Jisql.java 53a6ca4f 
>   
> kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSMetricUtil.java 
> 1527681a 
>   
> plugin-solr/src/main/java/org/apache/ranger/services/solr/client/ServiceSolrClient.java
>  5875a298 
>   security-admin/src/main/java/org/apache/ranger/biz/AssetMgr.java a53d46af 
>   security-admin/src/main/java/org/apache/ranger/biz/UserMgr.java 272dec40 
>   security-admin/src/main/java/org/apache/ranger/common/RESTErrorUtil.java 
> 967804ad 
>   security-admin/src/main/java/org/apache/ranger/common/StringUtil.java 
> 045e07c4 
>   
> security-admin/src/main/java/org/apache/ranger/patch/cliutil/MetricUtil.java 
> d1ab0d09 
>   security-admin/src/main/java/org/apache/ranger/service/UserService.java 
> 3fb279e9 
>   
> security-admin/src/main/java/org/apache/ranger/service/filter/RangerRESTAPIFilter.java
>  551f824b 
>   security-admin/src/main/java/org/apache/ranger/util/RangerRestUtil.java 
> fe7a53e5 
>   
> ugsync/src/main/java/org/apache/ranger/ldapusersync/process/LdapUserGroupBuilder.java
>  8dc147ca 
>   
> ugsync/src/main/java/org/apache/ranger/unixusersync/process/PolicyMgrUserGroupBuilder.java
>  ade2ee71 
> 
> 
> Diff: https://reviews.apache.org/r/65485/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Colm O hEigeartaigh
> 
>

[jira] [Commented] (RANGER-1970) Build fails - rat issues with storm-agent

2018-02-02 Thread Colm O hEigeartaigh (JIRA)


[ 
https://issues.apache.org/jira/browse/RANGER-1970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350743#comment-16350743
 ] 

Colm O hEigeartaigh commented on RANGER-1970:
-

Could you try deleting those files + apply this patch locally? It should fix 
the problem...

> Build fails - rat issues with storm-agent
> -
>
> Key: RANGER-1970
> URL: https://issues.apache.org/jira/browse/RANGER-1970
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Reporter: Nigel Jones
>Assignee: Colm O hEigeartaigh
>Priority: Major
> Fix For: 1.0.0
>
> Attachments: RANGER-1970.patch
>
>
> Cloned ranger source into a clean directory (java is 8 u162 (oracle), OS is 
> MacOS, maven is 3.5.2), , tried to build using the incantation recommended in 
> the ranger docs:
> -DskipTests=false clean compile package install assembly:assembly
> Version info:
> 5:23 $ mvn --version
> Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 
> 2017-10-18T08:58:13+01:00)
> Maven home: /usr/local/Cellar/maven/3.5.2/libexec
> Java version: 1.8.0_162, vendor: Oracle Corporation
> Java home: 
> /Library/Java/JavaVirtualMachines/jdk1.8.0_162.jdk/Contents/Home/jre
> Default locale: en_GB, platform encoding: UTF-8
> OS name: "mac os x", version: "10.13.3", arch: "x86_64", family: "mac"
> When doing this the build fails in 
> [INFO] Rat check: Summary of files. Unapproved: 2 unknown: 2 generated: 0 
> approved: 24 licence.
> 
> [INFO] Knox Security Plugin ... SUCCESS [04:50 
> min]
> [INFO] Storm Security Plugin .. FAILURE [ 18.102 
> s]
> [INFO] YARN Security Plugin ... SKIPPED
> ...
> [ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.11:check 
> (default) on project ranger-storm-plugin: Too many files with unapproved 
> license: 2 See RAT report in: 
> /Users/jonesn/IdeaProjects/ranger/storm-agent/target/rat.txt -> [Help 1]
>  
> Looking at the cause I see in the rat.txt:
> Unapproved licenses:
> /Users/jonesn/IdeaProjects/ranger/storm-agent/logs/workers-artifacts/word-count-1-1517582937/1024/worker.yaml
>  
> /Users/jonesn/IdeaProjects/ranger/storm-agent/logs/workers-artifacts/stormdev-2-1517582937/1027/worker.yaml
>  
> And the files have indeed just been created:
> 4:53 $ cd /Users/jonesn/IdeaProjects/ranger/storm-agent/logs/workers-artifacts
> ✔ ~/IdeaProjects/ranger/storm-agent/logs/workers-artifacts [master|…3]
> 14:53 $ ls
> stormdev-2-1517582937 word-count-1-1517582937
> ✔ ~/IdeaProjects/ranger/storm-agent/logs/workers-artifacts [master|…3]
> 14:53 $ ls -a
> . stormdev-2-1517582937
> .. word-count-1-1517582937
> ✔ ~/IdeaProjects/ranger/storm-agent/logs/workers-artifacts [master|…3]
>  
> So this looks like a build oversight with storm agent - either this needs 
> clearing up, or ignoring in rat config?
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (RANGER-1970) Build fails - rat issues with storm-agent

2018-02-02 Thread Colm O hEigeartaigh (JIRA)


 [ 
https://issues.apache.org/jira/browse/RANGER-1970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Colm O hEigeartaigh updated RANGER-1970:

Attachment: RANGER-1970.patch

> Build fails - rat issues with storm-agent
> -
>
> Key: RANGER-1970
> URL: https://issues.apache.org/jira/browse/RANGER-1970
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Reporter: Nigel Jones
>Assignee: Colm O hEigeartaigh
>Priority: Major
> Fix For: 1.0.0
>
> Attachments: RANGER-1970.patch
>
>
> Cloned ranger source into a clean directory (java is 8 u162 (oracle), OS is 
> MacOS, maven is 3.5.2), , tried to build using the incantation recommended in 
> the ranger docs:
> -DskipTests=false clean compile package install assembly:assembly
> Version info:
> 5:23 $ mvn --version
> Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 
> 2017-10-18T08:58:13+01:00)
> Maven home: /usr/local/Cellar/maven/3.5.2/libexec
> Java version: 1.8.0_162, vendor: Oracle Corporation
> Java home: 
> /Library/Java/JavaVirtualMachines/jdk1.8.0_162.jdk/Contents/Home/jre
> Default locale: en_GB, platform encoding: UTF-8
> OS name: "mac os x", version: "10.13.3", arch: "x86_64", family: "mac"
> When doing this the build fails in 
> [INFO] Rat check: Summary of files. Unapproved: 2 unknown: 2 generated: 0 
> approved: 24 licence.
> 
> [INFO] Knox Security Plugin ... SUCCESS [04:50 
> min]
> [INFO] Storm Security Plugin .. FAILURE [ 18.102 
> s]
> [INFO] YARN Security Plugin ... SKIPPED
> ...
> [ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.11:check 
> (default) on project ranger-storm-plugin: Too many files with unapproved 
> license: 2 See RAT report in: 
> /Users/jonesn/IdeaProjects/ranger/storm-agent/target/rat.txt -> [Help 1]
>  
> Looking at the cause I see in the rat.txt:
> Unapproved licenses:
> /Users/jonesn/IdeaProjects/ranger/storm-agent/logs/workers-artifacts/word-count-1-1517582937/1024/worker.yaml
>  
> /Users/jonesn/IdeaProjects/ranger/storm-agent/logs/workers-artifacts/stormdev-2-1517582937/1027/worker.yaml
>  
> And the files have indeed just been created:
> 4:53 $ cd /Users/jonesn/IdeaProjects/ranger/storm-agent/logs/workers-artifacts
> ✔ ~/IdeaProjects/ranger/storm-agent/logs/workers-artifacts [master|…3]
> 14:53 $ ls
> stormdev-2-1517582937 word-count-1-1517582937
> ✔ ~/IdeaProjects/ranger/storm-agent/logs/workers-artifacts [master|…3]
> 14:53 $ ls -a
> . stormdev-2-1517582937
> .. word-count-1-1517582937
> ✔ ~/IdeaProjects/ranger/storm-agent/logs/workers-artifacts [master|…3]
>  
> So this looks like a build oversight with storm agent - either this needs 
> clearing up, or ignoring in rat config?
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (RANGER-1970) Build fails - rat issues with storm-agent

2018-02-02 Thread Colm O hEigeartaigh (JIRA)


 [ 
https://issues.apache.org/jira/browse/RANGER-1970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Colm O hEigeartaigh reassigned RANGER-1970:
---

Assignee: Colm O hEigeartaigh

> Build fails - rat issues with storm-agent
> -
>
> Key: RANGER-1970
> URL: https://issues.apache.org/jira/browse/RANGER-1970
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Reporter: Nigel Jones
>Assignee: Colm O hEigeartaigh
>Priority: Major
>
> Cloned ranger source into a clean directory (java is 8 u162 (oracle), OS is 
> MacOS, maven is 3.5.2), , tried to build using the incantation recommended in 
> the ranger docs:
> -DskipTests=false clean compile package install assembly:assembly
> Version info:
> 5:23 $ mvn --version
> Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 
> 2017-10-18T08:58:13+01:00)
> Maven home: /usr/local/Cellar/maven/3.5.2/libexec
> Java version: 1.8.0_162, vendor: Oracle Corporation
> Java home: 
> /Library/Java/JavaVirtualMachines/jdk1.8.0_162.jdk/Contents/Home/jre
> Default locale: en_GB, platform encoding: UTF-8
> OS name: "mac os x", version: "10.13.3", arch: "x86_64", family: "mac"
> When doing this the build fails in 
> [INFO] Rat check: Summary of files. Unapproved: 2 unknown: 2 generated: 0 
> approved: 24 licence.
> 
> [INFO] Knox Security Plugin ... SUCCESS [04:50 
> min]
> [INFO] Storm Security Plugin .. FAILURE [ 18.102 
> s]
> [INFO] YARN Security Plugin ... SKIPPED
> ...
> [ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.11:check 
> (default) on project ranger-storm-plugin: Too many files with unapproved 
> license: 2 See RAT report in: 
> /Users/jonesn/IdeaProjects/ranger/storm-agent/target/rat.txt -> [Help 1]
>  
> Looking at the cause I see in the rat.txt:
> Unapproved licenses:
> /Users/jonesn/IdeaProjects/ranger/storm-agent/logs/workers-artifacts/word-count-1-1517582937/1024/worker.yaml
>  
> /Users/jonesn/IdeaProjects/ranger/storm-agent/logs/workers-artifacts/stormdev-2-1517582937/1027/worker.yaml
>  
> And the files have indeed just been created:
> 4:53 $ cd /Users/jonesn/IdeaProjects/ranger/storm-agent/logs/workers-artifacts
> ✔ ~/IdeaProjects/ranger/storm-agent/logs/workers-artifacts [master|…3]
> 14:53 $ ls
> stormdev-2-1517582937 word-count-1-1517582937
> ✔ ~/IdeaProjects/ranger/storm-agent/logs/workers-artifacts [master|…3]
> 14:53 $ ls -a
> . stormdev-2-1517582937
> .. word-count-1-1517582937
> ✔ ~/IdeaProjects/ranger/storm-agent/logs/workers-artifacts [master|…3]
>  
> So this looks like a build oversight with storm agent - either this needs 
> clearing up, or ignoring in rat config?
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (RANGER-1966) Policy engine initialization does not create context enrichers in some cases

2018-02-02 Thread Velmurugan Periasamy (JIRA)


 [ 
https://issues.apache.org/jira/browse/RANGER-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Velmurugan Periasamy updated RANGER-1966:
-
Fix Version/s: (was: master)
   1.0.0

> Policy engine initialization does not create context enrichers in some cases
> 
>
> Key: RANGER-1966
> URL: https://issues.apache.org/jira/browse/RANGER-1966
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: 1.0.0, 0.7.1
>Reporter: Abhay Kulkarni
>Assignee: Nigel Jones
>Priority: Major
> Fix For: 1.0.0, 0.7.2
>
>
> If no access policy evaluators are created, then policy engine does not 
> create context enrichers. Context enrichers need to be created if any type of 
> evaluators exist in the policy engine.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Review Request 65485: RANGER-1971 - Switch to use for-each loops

2018-02-02 Thread Colm O hEigeartaigh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65485/
---

Review request for ranger.


Bugs: RANGER-1971
https://issues.apache.org/jira/browse/RANGER-1971


Repository: ranger


Description
---

We should switch to use for-each loops in the code, as it is more readable and 
concise than using an index.


Diffs
-

  
agents-audit/src/main/java/org/apache/ranger/audit/provider/hdfs/HdfsLogDestination.java
 065e8b07 
  agents-audit/src/main/java/org/apache/ranger/audit/queue/AuditFileSpool.java 
9abd99f5 
  
agents-common/src/main/java/org/apache/ranger/plugin/geo/GeolocationMetadata.java
 d27a0308 
  
agents-common/src/main/java/org/apache/ranger/plugin/geo/RangerGeolocationData.java
 99d6027f 
  
agents-common/src/test/java/org/apache/ranger/plugin/policyengine/TestCacheMap.java
 5f39b224 
  credentialbuilder/src/main/java/org/apache/ranger/credentialapi/buildks.java 
b8bdb6fa 
  
hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/RangerAuthorizationCoprocessor.java
 e30f7957 
  
hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java
 fa84b138 
  jisql/src/main/java/org/apache/util/sql/Jisql.java 53a6ca4f 
  kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSMetricUtil.java 
1527681a 
  
plugin-solr/src/main/java/org/apache/ranger/services/solr/client/ServiceSolrClient.java
 5875a298 
  security-admin/src/main/java/org/apache/ranger/biz/AssetMgr.java a53d46af 
  security-admin/src/main/java/org/apache/ranger/biz/UserMgr.java 272dec40 
  security-admin/src/main/java/org/apache/ranger/common/RESTErrorUtil.java 
967804ad 
  security-admin/src/main/java/org/apache/ranger/common/StringUtil.java 
045e07c4 
  security-admin/src/main/java/org/apache/ranger/patch/cliutil/MetricUtil.java 
d1ab0d09 
  security-admin/src/main/java/org/apache/ranger/service/UserService.java 
3fb279e9 
  
security-admin/src/main/java/org/apache/ranger/service/filter/RangerRESTAPIFilter.java
 551f824b 
  security-admin/src/main/java/org/apache/ranger/util/RangerRestUtil.java 
fe7a53e5 
  
ugsync/src/main/java/org/apache/ranger/ldapusersync/process/LdapUserGroupBuilder.java
 8dc147ca 
  
ugsync/src/main/java/org/apache/ranger/unixusersync/process/PolicyMgrUserGroupBuilder.java
 ade2ee71 


Diff: https://reviews.apache.org/r/65485/diff/1/


Testing
---


Thanks,

Colm O hEigeartaigh

[jira] [Updated] (RANGER-1971) Switch to use for-each loops

2018-02-02 Thread Colm O hEigeartaigh (JIRA)


 [ 
https://issues.apache.org/jira/browse/RANGER-1971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Colm O hEigeartaigh updated RANGER-1971:

Attachment: 0001-RANGER-1971-Switch-to-use-for-each-loops.patch

> Switch to use for-each loops
> 
>
> Key: RANGER-1971
> URL: https://issues.apache.org/jira/browse/RANGER-1971
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Reporter: Colm O hEigeartaigh
>Assignee: Colm O hEigeartaigh
>Priority: Minor
> Fix For: 1.0.0
>
> Attachments: 0001-RANGER-1971-Switch-to-use-for-each-loops.patch
>
>
> We should switch to use for-each loops in the code, as it is more readable 
> and concise than using an index.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (RANGER-1971) Switch to use for-each loops

2018-02-02 Thread Colm O hEigeartaigh (JIRA)

Colm O hEigeartaigh created RANGER-1971:
---

 Summary: Switch to use for-each loops
 Key: RANGER-1971
 URL: https://issues.apache.org/jira/browse/RANGER-1971
 Project: Ranger
  Issue Type: Improvement
  Components: plugins
Reporter: Colm O hEigeartaigh
Assignee: Colm O hEigeartaigh
 Fix For: 1.0.0


We should switch to use for-each loops in the code, as it is more readable and 
concise than using an index.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (RANGER-1970) Build fails - rat issues with storm-agent

2018-02-02 Thread Nigel Jones (JIRA)

Nigel Jones created RANGER-1970:
---

 Summary: Build fails - rat issues with storm-agent
 Key: RANGER-1970
 URL: https://issues.apache.org/jira/browse/RANGER-1970
 Project: Ranger
  Issue Type: Bug
  Components: Ranger
Reporter: Nigel Jones


Cloned ranger source into a clean directory (java is 8 u162 (oracle), OS is 
MacOS, maven is 3.5.2), , tried to build using the incantation recommended in 
the ranger docs:

-DskipTests=false clean compile package install assembly:assembly

Version info:

5:23 $ mvn --version
Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 
2017-10-18T08:58:13+01:00)
Maven home: /usr/local/Cellar/maven/3.5.2/libexec
Java version: 1.8.0_162, vendor: Oracle Corporation
Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_162.jdk/Contents/Home/jre
Default locale: en_GB, platform encoding: UTF-8
OS name: "mac os x", version: "10.13.3", arch: "x86_64", family: "mac"

When doing this the build fails in 

[INFO] Rat check: Summary of files. Unapproved: 2 unknown: 2 generated: 0 
approved: 24 licence.



[INFO] Knox Security Plugin ... SUCCESS [04:50 min]
[INFO] Storm Security Plugin .. FAILURE [ 18.102 s]
[INFO] YARN Security Plugin ... SKIPPED

...

[ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.11:check 
(default) on project ranger-storm-plugin: Too many files with unapproved 
license: 2 See RAT report in: 
/Users/jonesn/IdeaProjects/ranger/storm-agent/target/rat.txt -> [Help 1]

 

Looking at the cause I see in the rat.txt:


Unapproved licenses:

/Users/jonesn/IdeaProjects/ranger/storm-agent/logs/workers-artifacts/word-count-1-1517582937/1024/worker.yaml
 
/Users/jonesn/IdeaProjects/ranger/storm-agent/logs/workers-artifacts/stormdev-2-1517582937/1027/worker.yaml

 

And the files have indeed just been created:

4:53 $ cd /Users/jonesn/IdeaProjects/ranger/storm-agent/logs/workers-artifacts
✔ ~/IdeaProjects/ranger/storm-agent/logs/workers-artifacts [master|…3]
14:53 $ ls
stormdev-2-1517582937 word-count-1-1517582937
✔ ~/IdeaProjects/ranger/storm-agent/logs/workers-artifacts [master|…3]
14:53 $ ls -a
. stormdev-2-1517582937
.. word-count-1-1517582937
✔ ~/IdeaProjects/ranger/storm-agent/logs/workers-artifacts [master|…3]

 

So this looks like a build oversight with storm agent - either this needs 
clearing up, or ignoring in rat config?

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Review Request 65484: RANGER-1969 - Fix failing Kafka tests with latest maven/JVM

2018-02-02 Thread Colm O hEigeartaigh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65484/
---

Review request for ranger.


Bugs: RANGER-1969
https://issues.apache.org/jira/browse/RANGER-1969


Repository: ranger


Description
---

The Kafka SASL/SSL test is crashing for me with the latest Maven/JVM. Upgrading 
the surefire plugin solves the problem.


Diffs
-

  pom.xml 255b02aa 


Diff: https://reviews.apache.org/r/65484/diff/1/


Testing
---


Thanks,

Colm O hEigeartaigh

[jira] [Updated] (RANGER-1969) Fix failing Kafka tests with latest maven/JVM

2018-02-02 Thread Colm O hEigeartaigh (JIRA)


 [ 
https://issues.apache.org/jira/browse/RANGER-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Colm O hEigeartaigh updated RANGER-1969:

Attachment: 0001-RANGER-1969-Fix-failing-Kafka-tests-with-latest-mave.patch

> Fix failing Kafka tests with latest maven/JVM
> -
>
> Key: RANGER-1969
> URL: https://issues.apache.org/jira/browse/RANGER-1969
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Reporter: Colm O hEigeartaigh
>Assignee: Colm O hEigeartaigh
>Priority: Major
> Fix For: 1.0.0
>
> Attachments: 
> 0001-RANGER-1969-Fix-failing-Kafka-tests-with-latest-mave.patch
>
>
> The Kafka SASL/SSL test is crashing for me with the latest Maven/JVM. 
> Upgrading the surefire plugin solves the problem.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (RANGER-1969) Fix failing Kafka tests with latest maven/JVM

2018-02-02 Thread Colm O hEigeartaigh (JIRA)

Colm O hEigeartaigh created RANGER-1969:
---

 Summary: Fix failing Kafka tests with latest maven/JVM
 Key: RANGER-1969
 URL: https://issues.apache.org/jira/browse/RANGER-1969
 Project: Ranger
  Issue Type: Improvement
  Components: plugins
Reporter: Colm O hEigeartaigh
Assignee: Colm O hEigeartaigh
 Fix For: 1.0.0


The Kafka SASL/SSL test is crashing for me with the latest Maven/JVM. Upgrading 
the surefire plugin solves the problem.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

servicedef validation - implClass

2018-02-02 Thread Nigel Jones

We're currently writing a Ranger plugin for an open source virtual database 
driver known as ‘GaianDB’.

We have the basics of a plugin working, but today a colleague tried to setup 
our plugin, and on their ranger install they could not create an instance of 
our ranger service, hitting the error 

"Gaian failed to find service class 
org.apache.ranger.services.gaiandb.RangerServiceGaian. Resource lookup will not 
be available. Please make sure plugin jar is in the correct place.”

I had expected this to be a warning (fair enough), but it actually prevents the 
UI from saving the service instance, and so Gaian related policies cannot be 
created.

They got this on I think a) an old 0.6.x build, and 0.7.1. 
Meanwhile I had been using a HDP 2.6.3 setup, which comes with 0.7.0 and with 
the same service definition did NOT of course hit this error.

We haven’t actually deployed any plugin code to the ranger server (yet) since 
we have not implemented the resource lookup capability. We had though specified 
an implClass in the servicedef. It’s a later task :-)

Looking at the ranger code in ServiceMgr.java it seems as if an empty string 
should cause ranger to use a default class. We tried this, as well as 
specifying the org.apache.ranger.plugin.service.RangerDefaultService class, but 
had the same error in both cases. I was though looking at code in master and 
see there have been a few changes

Any suggestions as to how to persuade 0.7.0 to load the servicedef (other than 
implement the class required, albeit a no-op one…?), or an explanation as to 
why we might see different behaviour?

I’m setting up a ranger install off master to debug/check latest code….

For info the servicedef begins:
{
  "id":99,
  "name": "gaian",
  "implClass": "org.apache.ranger.services.gaiandb.RangerServiceGaian",
  "label": "Gaian",
  "description": "Gaian",
  "options": {
"enableDenyAndExceptionsInPolicies": "true"
  },
"guid": "86d10748-e4fc-442b-8991-f6a727054ece",

  "resources": [
The full version can be found in 
https://issues.apache.org/jira/browse/RANGER-1699 
 - see the latest 
attachment. Not as a patch as currently getting things working in a diff. Build 
tree

If it seems like a bug (rather than a newbie error or specific to hdp) let me 
know and I’ll open a jira!

Many thanks
Nigel.

[jira] [Commented] (RANGER-1699) gaiandb: Create initial plugin configuration file that can be deployed to ranger

2018-02-02 Thread Nigel Jones (JIRA)


[ 
https://issues.apache.org/jira/browse/RANGER-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350391#comment-16350391
 ] 

Nigel Jones commented on RANGER-1699:
-

Added current servicedef (not currently a patch as restructuring build)

> gaiandb: Create initial plugin configuration file that can be deployed to 
> ranger
> 
>
> Key: RANGER-1699
> URL: https://issues.apache.org/jira/browse/RANGER-1699
> Project: Ranger
>  Issue Type: Sub-task
>  Components: plugins
>Reporter: Nigel Jones
>Priority: Major
> Attachments: 0001-adding-gaiandb-serviceDef.patch, 
> ranger-servicedef-gaian.json
>
>
> Create initial configuration file for the gaiandb plugin for ranger which 
> will define policies, resources.
> Additional jiras will be opened to add extra capabilities later on. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (RANGER-1699) gaiandb: Create initial plugin configuration file that can be deployed to ranger

2018-02-02 Thread Nigel Jones (JIRA)


 [ 
https://issues.apache.org/jira/browse/RANGER-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nigel Jones updated RANGER-1699:

Attachment: ranger-servicedef-gaian.json

> gaiandb: Create initial plugin configuration file that can be deployed to 
> ranger
> 
>
> Key: RANGER-1699
> URL: https://issues.apache.org/jira/browse/RANGER-1699
> Project: Ranger
>  Issue Type: Sub-task
>  Components: plugins
>Reporter: Nigel Jones
>Priority: Major
> Attachments: 0001-adding-gaiandb-serviceDef.patch, 
> ranger-servicedef-gaian.json
>
>
> Create initial configuration file for the gaiandb plugin for ranger which 
> will define policies, resources.
> Additional jiras will be opened to add extra capabilities later on. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 65484: RANGER-1969 - Fix failing Kafka tests with latest maven/JVM

Re: servicedef validation - implClass

Re: Review Request 65485: RANGER-1971 - Switch to use for-each loops

[jira] [Commented] (RANGER-1970) Build fails - rat issues with storm-agent

[jira] [Updated] (RANGER-1970) Build fails - rat issues with storm-agent

[jira] [Assigned] (RANGER-1970) Build fails - rat issues with storm-agent

[jira] [Updated] (RANGER-1966) Policy engine initialization does not create context enrichers in some cases

Review Request 65485: RANGER-1971 - Switch to use for-each loops

[jira] [Updated] (RANGER-1971) Switch to use for-each loops

[jira] [Created] (RANGER-1971) Switch to use for-each loops

[jira] [Created] (RANGER-1970) Build fails - rat issues with storm-agent

Review Request 65484: RANGER-1969 - Fix failing Kafka tests with latest maven/JVM

[jira] [Updated] (RANGER-1969) Fix failing Kafka tests with latest maven/JVM

[jira] [Created] (RANGER-1969) Fix failing Kafka tests with latest maven/JVM

servicedef validation - implClass

[jira] [Commented] (RANGER-1699) gaiandb: Create initial plugin configuration file that can be deployed to ranger

[jira] [Updated] (RANGER-1699) gaiandb: Create initial plugin configuration file that can be deployed to ranger

17 matches

Site Navigation

Mail list logo

Footer information