[jira] [Resolved] (RANGER-1488) Create Ranger plugin for gaiandb
[ https://issues.apache.org/jira/browse/RANGER-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones resolved RANGER-1488. - Resolution: Won't Do > Create Ranger plugin for gaiandb > > > Key: RANGER-1488 > URL: https://issues.apache.org/jira/browse/RANGER-1488 > Project: Ranger > Issue Type: New Feature > Components: plugins >Reporter: Nigel Jones >Assignee: Nigel Jones >Priority: Major > Labels: VirtualDataConnector > Attachments: GaianDebuggingResearch.docx > > > GaianDB is a distributed, federated database built on Apache Derby. > The documentation is at > https://github.com/gaiandb/gaiandb/blob/master/README.md - navigate up for > the source. > As part of a Virtual Connector Project based on Atlas we are using gaianDB to > provide a virtualization layer. We need to control access to underlying > resources and will be building a Ranger plugin for gaiandb to support this. > GaianDB already has support for a form of policy plugin which allows the SQL > to be intercepted. > It is unclear if this code will sit > - external to ranger/atlas ie in the gaianDB sources > - as a sample in the Atlas project (so that we collate what is needed for > the virtual connector project) > - as a sample, or component in the Ranger project > However it seems helpful to describe the proposal here and get > feedback/dialogue in the ranger community :-) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (RANGER-1488) Create Ranger plugin for gaiandb
[ https://issues.apache.org/jira/browse/RANGER-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741249#comment-16741249 ] Nigel Jones commented on RANGER-1488: - The gaian plugin is now to be found in the ODPi Egeria project. See [https://github.com/odpi/egeria/tree/master/open-metadata-implementation/adapters/governance-engines-plugins/gaian-ranger-plugin] for the plugin, as well as [https://github.com/odpi/egeria/tree/master/open-metadata-implementation/adapters/authentication-plugins/gaian-impersonation] for impersonation support I will close this Jira as no further activity is expected here in the near term. > Create Ranger plugin for gaiandb > > > Key: RANGER-1488 > URL: https://issues.apache.org/jira/browse/RANGER-1488 > Project: Ranger > Issue Type: New Feature > Components: plugins >Reporter: Nigel Jones >Assignee: Nigel Jones >Priority: Major > Labels: VirtualDataConnector > Attachments: GaianDebuggingResearch.docx > > > GaianDB is a distributed, federated database built on Apache Derby. > The documentation is at > https://github.com/gaiandb/gaiandb/blob/master/README.md - navigate up for > the source. > As part of a Virtual Connector Project based on Atlas we are using gaianDB to > provide a virtualization layer. We need to control access to underlying > resources and will be building a Ranger plugin for gaiandb to support this. > GaianDB already has support for a form of policy plugin which allows the SQL > to be intercepted. > It is unclear if this code will sit > - external to ranger/atlas ie in the gaianDB sources > - as a sample in the Atlas project (so that we collate what is needed for > the virtual connector project) > - as a sample, or component in the Ranger project > However it seems helpful to describe the proposal here and get > feedback/dialogue in the ranger community :-) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (RANGER-2254) Ranger haven't building by "build_ranger_using_docker.sh"
[ https://issues.apache.org/jira/browse/RANGER-2254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686981#comment-16686981 ] Nigel Jones commented on RANGER-2254: - If it helps, my current docker scripts which botj build ranger, and then go onto configuring it (in a very basic way, with admin server only) are at [https://github.com/odpi/egeria/tree/master/open-metadata-resources/open-metadata-deployment/docker/ranger] The intent is to automate deployment of a demo scenario using the egeria open metadata project we're working on (which also will use atlas). I was hoping to look at how we could add some ranger docker images that could be deployed for use in future, but I'm still in the stages of getting the basics working. This is now focussed around using k8s too. Anyway I struggled a little recently with some of the build issues around packaging, but the above script now does seem to work - so the first part may be useful for you? Nigel. > Ranger haven't building by "build_ranger_using_docker.sh" > - > > Key: RANGER-2254 > URL: https://issues.apache.org/jira/browse/RANGER-2254 > Project: Ranger > Issue Type: Bug > Components: audit, Ranger >Affects Versions: 1.2.0 > Environment: docker >Reporter: Leonid >Priority: Major > Labels: build > Fix For: 1.2.0 > > Attachments: first.log, second.log > > > I have tried to build Ranger 1.2.0 with > {code:java} > sh ./build_ranger_using_docker.sh mvn clean compile package install > assembly:assembly > {code} > and had gotten this: > {code:java} > [ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.11:check > (default) on project ranger: Too many files with unapproved license: 22 See > RAT report in: /ranger/target/rat.txt -> [Help 1] > org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute > goal org.apache.rat:apache-rat-plugin:0.11:check (default) on project ranger: > Too many files with unapproved license: 22 See RAT report in: > /ranger/target/rat.txt > {code} > if I have tried use *_-Drat.skip=true_* I had gotten many errors in Audit > Component like *_error: cannot find symbol_* -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (RANGER-1699) gaiandb: Create initial plugin configuration file that can be deployed to ranger
[ https://issues.apache.org/jira/browse/RANGER-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones closed RANGER-1699. --- > gaiandb: Create initial plugin configuration file that can be deployed to > ranger > > > Key: RANGER-1699 > URL: https://issues.apache.org/jira/browse/RANGER-1699 > Project: Ranger > Issue Type: Sub-task > Components: plugins >Reporter: Nigel Jones >Priority: Major > Attachments: 0001-adding-gaiandb-serviceDef.patch, > ranger-servicedef-gaian.json > > > Create initial configuration file for the gaiandb plugin for ranger which > will define policies, resources. > Additional jiras will be opened to add extra capabilities later on. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (RANGER-1701) gaiandb: simple resource based policy
[ https://issues.apache.org/jira/browse/RANGER-1701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones closed RANGER-1701. --- > gaiandb: simple resource based policy > - > > Key: RANGER-1701 > URL: https://issues.apache.org/jira/browse/RANGER-1701 > Project: Ranger > Issue Type: Sub-task > Components: plugins >Reporter: Nigel Jones >Priority: Major > > Get the gaiandb plugin to a state where a simple resource based policy is > operational (permit/deny), and the ranger gui can be used to define.. and the > plugin will then implement this -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (RANGER-1703) gaiandb - add data masking (similar to hive) for resource based policy
[ https://issues.apache.org/jira/browse/RANGER-1703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones closed RANGER-1703. --- > gaiandb - add data masking (similar to hive) for resource based policy > -- > > Key: RANGER-1703 > URL: https://issues.apache.org/jira/browse/RANGER-1703 > Project: Ranger > Issue Type: Sub-task > Components: plugins >Reporter: Nigel Jones >Priority: Major > > Support data masking in gaindb plugin. Initially implement for a resource > based policy -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (RANGER-1909) Build failure in Kafka Security plugin Unit Tests
[ https://issues.apache.org/jira/browse/RANGER-1909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones closed RANGER-1909. --- > Build failure in Kafka Security plugin Unit Tests > - > > Key: RANGER-1909 > URL: https://issues.apache.org/jira/browse/RANGER-1909 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Nigel Jones >Priority: Major > Attachments: ranger-20171124a.log > > > Java 1.8 (151, openjdk) > Maven 3.5.0 > Build fails at > [INFO] KAFKA Security Plugin .. FAILURE [22:07 > min] > [INFO] Running > org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerGSSTest > log4j:WARN No appenders could be found for logger > (org.apache.kerby.kerberos.kerb.identity.backend.AbstractIdentityBackend). > log4j:WARN Please initialize the log4j system properly. > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more > info. > [ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: > 1,120.51 s <<< FAILURE! - in > org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerGSSTest > [ERROR] > testAuthorizedRead(org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerGSSTest) > Time elapsed: 1,109.032 s <<< FAILURE! > java.lang.AssertionError > at > org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerGSSTest.testAuthorizedRead(KafkaRangerAuthorizerGSSTest.java:243) > [INFO] Running > org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerTest > log4j:WARN No appenders could be found for logger > (org.apache.zookeeper.server.ZooKeeperServerMain). > log4j:WARN Please initialize the log4j system properly. > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more > info. > [INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: > 114.827 s - in > org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerTest > [INFO] Running > org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerSASLSSLTest > log4j:WARN No appenders could be found for logger > (org.apache.zookeeper.server.ZooKeeperServerMain). > log4j:WARN Please initialize the log4j system properly. > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more > info. > [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 72.214 > s - in > org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerSASLSSLTest > [INFO] > [INFO] Results: > [INFO] > [ERROR] Failures: > [ERROR] KafkaRangerAuthorizerGSSTest.testAuthorizedRead:243 > [INFO] > [ERROR] Tests run: 8, Failures: 1, Errors: 0, Skipped: 0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (RANGER-1850) Impersonation/proxy user support for gaiandb ranger plugin
[ https://issues.apache.org/jira/browse/RANGER-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones closed RANGER-1850. --- > Impersonation/proxy user support for gaiandb ranger plugin > -- > > Key: RANGER-1850 > URL: https://issues.apache.org/jira/browse/RANGER-1850 > Project: Ranger > Issue Type: Sub-task > Components: plugins >Reporter: Nigel Jones >Priority: Major > Attachments: GaianDBAuth.docx > > > Applications/users could connect to gaianDB using their own authentication > information - for example userid/password in the simple case. Here the ranger > plugin will use that id for policy checks. > However in a multi tiered architecture a service id (aka non personal > account) may be used, and somehow the user to be impersonated is passed via > an additional property. This has a number of implications to the system > configuration, derby/gaiandb configuration & the plugin implementation. > Opening this Jira as a placeholder and will add a document soon (++days) on > the same to capture some of the discussion around this area in recent days. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (RANGER-1850) Impersonation/proxy user support for gaiandb ranger plugin
[ https://issues.apache.org/jira/browse/RANGER-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones resolved RANGER-1850. - Resolution: Fixed > Impersonation/proxy user support for gaiandb ranger plugin > -- > > Key: RANGER-1850 > URL: https://issues.apache.org/jira/browse/RANGER-1850 > Project: Ranger > Issue Type: Sub-task > Components: plugins >Reporter: Nigel Jones >Priority: Major > Attachments: GaianDBAuth.docx > > > Applications/users could connect to gaianDB using their own authentication > information - for example userid/password in the simple case. Here the ranger > plugin will use that id for policy checks. > However in a multi tiered architecture a service id (aka non personal > account) may be used, and somehow the user to be impersonated is passed via > an additional property. This has a number of implications to the system > configuration, derby/gaiandb configuration & the plugin implementation. > Opening this Jira as a placeholder and will add a document soon (++days) on > the same to capture some of the discussion around this area in recent days. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (RANGER-1702) gaiandb: tag based policies
[ https://issues.apache.org/jira/browse/RANGER-1702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones reopened RANGER-1702: - > gaiandb: tag based policies > --- > > Key: RANGER-1702 > URL: https://issues.apache.org/jira/browse/RANGER-1702 > Project: Ranger > Issue Type: Sub-task > Components: plugins >Reporter: Nigel Jones >Priority: Major > > Update plugin to allow tag based policies to be used for simple permit/deny -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (RANGER-1702) gaiandb: tag based policies
[ https://issues.apache.org/jira/browse/RANGER-1702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones resolved RANGER-1702. - Resolution: Fixed > gaiandb: tag based policies > --- > > Key: RANGER-1702 > URL: https://issues.apache.org/jira/browse/RANGER-1702 > Project: Ranger > Issue Type: Sub-task > Components: plugins >Reporter: Nigel Jones >Priority: Major > > Update plugin to allow tag based policies to be used for simple permit/deny -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (RANGER-1703) gaiandb - add data masking (similar to hive) for resource based policy
[ https://issues.apache.org/jira/browse/RANGER-1703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones resolved RANGER-1703. - Resolution: Fixed > gaiandb - add data masking (similar to hive) for resource based policy > -- > > Key: RANGER-1703 > URL: https://issues.apache.org/jira/browse/RANGER-1703 > Project: Ranger > Issue Type: Sub-task > Components: plugins >Reporter: Nigel Jones >Priority: Major > > Support data masking in gaindb plugin. Initially implement for a resource > based policy -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (RANGER-1700) gaiandb: Integrate with ranger libraries - plugin builds, initializes
[ https://issues.apache.org/jira/browse/RANGER-1700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones resolved RANGER-1700. - Resolution: Fixed > gaiandb: Integrate with ranger libraries - plugin builds, initializes > - > > Key: RANGER-1700 > URL: https://issues.apache.org/jira/browse/RANGER-1700 > Project: Ranger > Issue Type: Sub-task > Components: plugins >Reporter: Nigel Jones >Priority: Major > Attachments: 0001-adding-gaianDB-connection-and-resource-manage.patch > > > Get the plugin to a state where it will initialize with a simple > configuration and can be deployed to gaianDB. It may not actually action any > policies at this state -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (RANGER-1699) gaiandb: Create initial plugin configuration file that can be deployed to ranger
[ https://issues.apache.org/jira/browse/RANGER-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones resolved RANGER-1699. - Resolution: Fixed > gaiandb: Create initial plugin configuration file that can be deployed to > ranger > > > Key: RANGER-1699 > URL: https://issues.apache.org/jira/browse/RANGER-1699 > Project: Ranger > Issue Type: Sub-task > Components: plugins >Reporter: Nigel Jones >Priority: Major > Attachments: 0001-adding-gaiandb-serviceDef.patch, > ranger-servicedef-gaian.json > > > Create initial configuration file for the gaiandb plugin for ranger which > will define policies, resources. > Additional jiras will be opened to add extra capabilities later on. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (RANGER-1701) gaiandb: simple resource based policy
[ https://issues.apache.org/jira/browse/RANGER-1701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones resolved RANGER-1701. - > gaiandb: simple resource based policy > - > > Key: RANGER-1701 > URL: https://issues.apache.org/jira/browse/RANGER-1701 > Project: Ranger > Issue Type: Sub-task > Components: plugins >Reporter: Nigel Jones >Priority: Major > > Get the gaiandb plugin to a state where a simple resource based policy is > operational (permit/deny), and the ranger gui can be used to define.. and the > plugin will then implement this -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (RANGER-1186) Ranger Source: eclipse
[ https://issues.apache.org/jira/browse/RANGER-1186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones closed RANGER-1186. --- > Ranger Source: eclipse > -- > > Key: RANGER-1186 > URL: https://issues.apache.org/jira/browse/RANGER-1186 > Project: Ranger > Issue Type: Bug > Components: documentation >Affects Versions: 0.7.0 > Environment: eclipse neon, ubuntu 16.10 >Reporter: Nigel Jones >Assignee: Pradeep Agrawal >Priority: Major > Labels: VirtualDataConnector > Attachments: RANGER-1186_Ranger_Source_Eclipse.pdf, ranger_me0.PNG, > ranger_me1.PNG > > > I can happily build ranger in my Ubuntu 16.10 environment with > - open jdk 1.8.0_102 > - maven 3.3.9 > I also have eclipse neon installed in this environment > The build/setup instructions at > http://ranger.apache.org/quick_start_guide.html have minimal instructions for > eclipse. > I imported the top level ranger directory using the m2e plugin as provided in > neon. > This failed with numerous errors > Is anyone actively building with eclipse? If so can we update the website > with current setup information? > If not I'll use this JIRA to record the issues & hope to address them & > update the docs ;-) > Screenshots to follow -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (RANGER-1331) Ranger build for windows not working
[ https://issues.apache.org/jira/browse/RANGER-1331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones resolved RANGER-1331. - Resolution: Cannot Reproduce > Ranger build for windows not working > > > Key: RANGER-1331 > URL: https://issues.apache.org/jira/browse/RANGER-1331 > Project: Ranger > Issue Type: Improvement >Reporter: Nigel Jones >Priority: Major > Attachments: RangerWinBuild1.log > > > The quick start guide at http://ranger.apache.org/quick_start_guide.html has > instuctions to build ranger which work fine on Ubuntu 16.04/openjdk 8 / maven > 3.3.9 > However I was interested in setting up a more productive IDE environment, and > given my laptop is running Windows 10 as the primary OS decided to follow the > instructions for using eclipse > HOWEVER unlike Ranger-1186 I tried this on *WINDOWS* rather than linux > Is this something anyone is doing successfully (even if skipping building > some of the linux native auth pieces)? > I then tried the build but it failed quickly on credential support > [INFO] ranger . SUCCESS [ 9.332 > s] > [INFO] Jdbc SQL Connector . SUCCESS [ 4.569 > s] > [INFO] Credential Support . FAILURE [ 9.579 > s] > Including full log > Could we list on that page the OSs people have had success with? > Are most using linux? > Anyone tried MacOS? > NOTE: edited from original to focus on the OS question rather than IDE -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (RANGER-1331) Ranger build for windows not working
[ https://issues.apache.org/jira/browse/RANGER-1331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones closed RANGER-1331. --- > Ranger build for windows not working > > > Key: RANGER-1331 > URL: https://issues.apache.org/jira/browse/RANGER-1331 > Project: Ranger > Issue Type: Improvement >Reporter: Nigel Jones >Priority: Major > Attachments: RangerWinBuild1.log > > > The quick start guide at http://ranger.apache.org/quick_start_guide.html has > instuctions to build ranger which work fine on Ubuntu 16.04/openjdk 8 / maven > 3.3.9 > However I was interested in setting up a more productive IDE environment, and > given my laptop is running Windows 10 as the primary OS decided to follow the > instructions for using eclipse > HOWEVER unlike Ranger-1186 I tried this on *WINDOWS* rather than linux > Is this something anyone is doing successfully (even if skipping building > some of the linux native auth pieces)? > I then tried the build but it failed quickly on credential support > [INFO] ranger . SUCCESS [ 9.332 > s] > [INFO] Jdbc SQL Connector . SUCCESS [ 4.569 > s] > [INFO] Credential Support . FAILURE [ 9.579 > s] > Including full log > Could we list on that page the OSs people have had success with? > Are most using linux? > Anyone tried MacOS? > NOTE: edited from original to focus on the OS question rather than IDE -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (RANGER-1908) Build (unit tests) failed in HBase Security Plugin
[ https://issues.apache.org/jira/browse/RANGER-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones closed RANGER-1908. --- > Build (unit tests) failed in HBase Security Plugin > -- > > Key: RANGER-1908 > URL: https://issues.apache.org/jira/browse/RANGER-1908 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Nigel Jones >Assignee: Nigel Jones >Priority: Major > Attachments: ranger-20171124a.log > > > MacOS > Java 1.8 (152, oracle) > Maven 3.5.0 > Build fails at > [INFO] HBase Security Plugin .. FAILURE [11:36 > min] > The failing test is: > [INFO] Running > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest > Formatting using clusterid: testClusterID > [ERROR] Tests run: 20, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: > 689.962 s <<< FAILURE! - in > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest > [ERROR] > testTagBasedTablePolicy(org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest) > Time elapsed: 611.879 s <<< ERROR! > org.apache.hadoop.hbase.exceptions.TimeoutIOException: > java.util.concurrent.TimeoutException: The procedure 18 is still running > at > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest.testTagBasedTablePolicy(HBaseRangerAuthorizationTest.java:780) > Caused by: java.util.concurrent.TimeoutException: The procedure 18 is still > running > at > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest.testTagBasedTablePolicy(HBaseRangerAuthorizationTest.java:780) > The next part of the test is ok: > [INFO] Running > org.apache.ranger.authorization.hbase.RangerAuthorizationFilterTest > [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.505 > s - in org.apache.ranger.authorization.hbase.RangerAuthorizationFilterTest -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (RANGER-1909) Build failure in Kafka Security plugin Unit Tests
[ https://issues.apache.org/jira/browse/RANGER-1909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones resolved RANGER-1909. - Resolution: Cannot Reproduce > Build failure in Kafka Security plugin Unit Tests > - > > Key: RANGER-1909 > URL: https://issues.apache.org/jira/browse/RANGER-1909 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Nigel Jones >Priority: Major > Attachments: ranger-20171124a.log > > > Java 1.8 (151, openjdk) > Maven 3.5.0 > Build fails at > [INFO] KAFKA Security Plugin .. FAILURE [22:07 > min] > [INFO] Running > org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerGSSTest > log4j:WARN No appenders could be found for logger > (org.apache.kerby.kerberos.kerb.identity.backend.AbstractIdentityBackend). > log4j:WARN Please initialize the log4j system properly. > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more > info. > [ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: > 1,120.51 s <<< FAILURE! - in > org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerGSSTest > [ERROR] > testAuthorizedRead(org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerGSSTest) > Time elapsed: 1,109.032 s <<< FAILURE! > java.lang.AssertionError > at > org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerGSSTest.testAuthorizedRead(KafkaRangerAuthorizerGSSTest.java:243) > [INFO] Running > org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerTest > log4j:WARN No appenders could be found for logger > (org.apache.zookeeper.server.ZooKeeperServerMain). > log4j:WARN Please initialize the log4j system properly. > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more > info. > [INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: > 114.827 s - in > org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerTest > [INFO] Running > org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerSASLSSLTest > log4j:WARN No appenders could be found for logger > (org.apache.zookeeper.server.ZooKeeperServerMain). > log4j:WARN Please initialize the log4j system properly. > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more > info. > [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 72.214 > s - in > org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerSASLSSLTest > [INFO] > [INFO] Results: > [INFO] > [ERROR] Failures: > [ERROR] KafkaRangerAuthorizerGSSTest.testAuthorizedRead:243 > [INFO] > [ERROR] Tests run: 8, Failures: 1, Errors: 0, Skipped: 0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (RANGER-1970) Build fails - rat issues with storm-agent
[ https://issues.apache.org/jira/browse/RANGER-1970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones closed RANGER-1970. --- > Build fails - rat issues with storm-agent > - > > Key: RANGER-1970 > URL: https://issues.apache.org/jira/browse/RANGER-1970 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Nigel Jones >Assignee: Colm O hEigeartaigh >Priority: Major > Fix For: 1.0.0 > > Attachments: RANGER-1970.patch > > > Cloned ranger source into a clean directory (java is 8 u162 (oracle), OS is > MacOS, maven is 3.5.2), , tried to build using the incantation recommended in > the ranger docs: > -DskipTests=false clean compile package install assembly:assembly > Version info: > 5:23 $ mvn --version > Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; > 2017-10-18T08:58:13+01:00) > Maven home: /usr/local/Cellar/maven/3.5.2/libexec > Java version: 1.8.0_162, vendor: Oracle Corporation > Java home: > /Library/Java/JavaVirtualMachines/jdk1.8.0_162.jdk/Contents/Home/jre > Default locale: en_GB, platform encoding: UTF-8 > OS name: "mac os x", version: "10.13.3", arch: "x86_64", family: "mac" > When doing this the build fails in > [INFO] Rat check: Summary of files. Unapproved: 2 unknown: 2 generated: 0 > approved: 24 licence. > > [INFO] Knox Security Plugin ... SUCCESS [04:50 > min] > [INFO] Storm Security Plugin .. FAILURE [ 18.102 > s] > [INFO] YARN Security Plugin ... SKIPPED > ... > [ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.11:check > (default) on project ranger-storm-plugin: Too many files with unapproved > license: 2 See RAT report in: > /Users/jonesn/IdeaProjects/ranger/storm-agent/target/rat.txt -> [Help 1] > > Looking at the cause I see in the rat.txt: > Unapproved licenses: > /Users/jonesn/IdeaProjects/ranger/storm-agent/logs/workers-artifacts/word-count-1-1517582937/1024/worker.yaml > > /Users/jonesn/IdeaProjects/ranger/storm-agent/logs/workers-artifacts/stormdev-2-1517582937/1027/worker.yaml > > And the files have indeed just been created: > 4:53 $ cd /Users/jonesn/IdeaProjects/ranger/storm-agent/logs/workers-artifacts > ✔ ~/IdeaProjects/ranger/storm-agent/logs/workers-artifacts [master|…3] > 14:53 $ ls > stormdev-2-1517582937 word-count-1-1517582937 > ✔ ~/IdeaProjects/ranger/storm-agent/logs/workers-artifacts [master|…3] > 14:53 $ ls -a > . stormdev-2-1517582937 > .. word-count-1-1517582937 > ✔ ~/IdeaProjects/ranger/storm-agent/logs/workers-artifacts [master|…3] > > So this looks like a build oversight with storm agent - either this needs > clearing up, or ignoring in rat config? > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (RANGER-1488) Create Ranger plugin for gaiandb
[ https://issues.apache.org/jira/browse/RANGER-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447791#comment-16447791 ] Nigel Jones commented on RANGER-1488: - Just an update ... the plugin has been coded and is working. The code is currently at [https://github.com/GaianRangerPlugin/ranger-gaian-plugin] and some associated issues in the github tracker It's not been pushed to ranger yet as we work through a few questions around gaian & where the source is managed, but I wanted to update people on progress. If interested do get in touch and am happy to explain/demo etc.. > Create Ranger plugin for gaiandb > > > Key: RANGER-1488 > URL: https://issues.apache.org/jira/browse/RANGER-1488 > Project: Ranger > Issue Type: New Feature > Components: plugins >Reporter: Nigel Jones >Assignee: Nigel Jones >Priority: Major > Labels: VirtualDataConnector > Attachments: GaianDebuggingResearch.docx > > > GaianDB is a distributed, federated database built on Apache Derby. > The documentation is at > https://github.com/gaiandb/gaiandb/blob/master/README.md - navigate up for > the source. > As part of a Virtual Connector Project based on Atlas we are using gaianDB to > provide a virtualization layer. We need to control access to underlying > resources and will be building a Ranger plugin for gaiandb to support this. > GaianDB already has support for a form of policy plugin which allows the SQL > to be intercepted. > It is unclear if this code will sit > - external to ranger/atlas ie in the gaianDB sources > - as a sample in the Atlas project (so that we collate what is needed for > the virtual connector project) > - as a sample, or component in the Ranger project > However it seems helpful to describe the proposal here and get > feedback/dialogue in the ranger community :-) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (RANGER-1970) Build fails - rat issues with storm-agent
Nigel Jones created RANGER-1970: --- Summary: Build fails - rat issues with storm-agent Key: RANGER-1970 URL: https://issues.apache.org/jira/browse/RANGER-1970 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Nigel Jones Cloned ranger source into a clean directory (java is 8 u162 (oracle), OS is MacOS, maven is 3.5.2), , tried to build using the incantation recommended in the ranger docs: -DskipTests=false clean compile package install assembly:assembly Version info: 5:23 $ mvn --version Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T08:58:13+01:00) Maven home: /usr/local/Cellar/maven/3.5.2/libexec Java version: 1.8.0_162, vendor: Oracle Corporation Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_162.jdk/Contents/Home/jre Default locale: en_GB, platform encoding: UTF-8 OS name: "mac os x", version: "10.13.3", arch: "x86_64", family: "mac" When doing this the build fails in [INFO] Rat check: Summary of files. Unapproved: 2 unknown: 2 generated: 0 approved: 24 licence. [INFO] Knox Security Plugin ... SUCCESS [04:50 min] [INFO] Storm Security Plugin .. FAILURE [ 18.102 s] [INFO] YARN Security Plugin ... SKIPPED ... [ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.11:check (default) on project ranger-storm-plugin: Too many files with unapproved license: 2 See RAT report in: /Users/jonesn/IdeaProjects/ranger/storm-agent/target/rat.txt -> [Help 1] Looking at the cause I see in the rat.txt: Unapproved licenses: /Users/jonesn/IdeaProjects/ranger/storm-agent/logs/workers-artifacts/word-count-1-1517582937/1024/worker.yaml /Users/jonesn/IdeaProjects/ranger/storm-agent/logs/workers-artifacts/stormdev-2-1517582937/1027/worker.yaml And the files have indeed just been created: 4:53 $ cd /Users/jonesn/IdeaProjects/ranger/storm-agent/logs/workers-artifacts ✔ ~/IdeaProjects/ranger/storm-agent/logs/workers-artifacts [master|…3] 14:53 $ ls stormdev-2-1517582937 word-count-1-1517582937 ✔ ~/IdeaProjects/ranger/storm-agent/logs/workers-artifacts [master|…3] 14:53 $ ls -a . stormdev-2-1517582937 .. word-count-1-1517582937 ✔ ~/IdeaProjects/ranger/storm-agent/logs/workers-artifacts [master|…3] So this looks like a build oversight with storm agent - either this needs clearing up, or ignoring in rat config? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (RANGER-1699) gaiandb: Create initial plugin configuration file that can be deployed to ranger
[ https://issues.apache.org/jira/browse/RANGER-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350391#comment-16350391 ] Nigel Jones commented on RANGER-1699: - Added current servicedef (not currently a patch as restructuring build) > gaiandb: Create initial plugin configuration file that can be deployed to > ranger > > > Key: RANGER-1699 > URL: https://issues.apache.org/jira/browse/RANGER-1699 > Project: Ranger > Issue Type: Sub-task > Components: plugins >Reporter: Nigel Jones >Priority: Major > Attachments: 0001-adding-gaiandb-serviceDef.patch, > ranger-servicedef-gaian.json > > > Create initial configuration file for the gaiandb plugin for ranger which > will define policies, resources. > Additional jiras will be opened to add extra capabilities later on. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (RANGER-1699) gaiandb: Create initial plugin configuration file that can be deployed to ranger
[ https://issues.apache.org/jira/browse/RANGER-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated RANGER-1699: Attachment: ranger-servicedef-gaian.json > gaiandb: Create initial plugin configuration file that can be deployed to > ranger > > > Key: RANGER-1699 > URL: https://issues.apache.org/jira/browse/RANGER-1699 > Project: Ranger > Issue Type: Sub-task > Components: plugins >Reporter: Nigel Jones >Priority: Major > Attachments: 0001-adding-gaiandb-serviceDef.patch, > ranger-servicedef-gaian.json > > > Create initial configuration file for the gaiandb plugin for ranger which > will define policies, resources. > Additional jiras will be opened to add extra capabilities later on. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (RANGER-1850) Impersonation/proxy user support for gaiandb ranger plugin
[ https://issues.apache.org/jira/browse/RANGER-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313281#comment-16313281 ] Nigel Jones commented on RANGER-1850: - I have an example flow on the 'proxy auth' tab (along bottom) of a few diagrams on draw.io - https://www.draw.io/#Hplanetf1%2Fgaiandb-policy-ranger%2Fmaster%2Fdoc%2FGaian%20Data%20Access%20Diagrams To answer * userid Ernie is coming in from the JDBC application into gaiandb. As part of that jdbc connect proxy-uid=gaiandb, proxy-pass=passw0rd is also sent * gaiandb/passw0rd is used by the proxy auth module to authenticate with the base gaiandb/derby authentication module * Once connected the application does indeed have access to all the gaiandb virtual tables in that there is no control over returned metadata (std jdbc metadata) as gaiandb doesn't support policy management there (I think it should in future). At a derby level gaiandb does have access to everything * If the ranger plugin is installed, then access requests to individual schemas, tables & columns will be controlled at that point (using user Ernie) * This difference between derby's base auth model using grants etc & what ranger does is not ideal... but requires significant work to better integrate gaiandb & it's policy model into the derby security manager * The userids configured to connect to the underlaying data sources that gaiandb is virtualizing are defined in the gaiandb configuration files (on the gaiandb code co-located with that data source typically). Whilst the user Ernie can access these data sources, he does not have direct access to the authentication details as such, or at least doesn't use them, that's the point of using gaiandb as the virtualization layer * A individual data source will have one set of credentials defined in the gaiandb configuration ,so regardless of which user connects the same credentials are used, so by definition they are NPAs One point that does surface from this is that access to the gaiandb stored procedures/derby tables that contain auth info for the underlying sources should be restricted. I am not sure if this is possible but will look into it. > Impersonation/proxy user support for gaiandb ranger plugin > -- > > Key: RANGER-1850 > URL: https://issues.apache.org/jira/browse/RANGER-1850 > Project: Ranger > Issue Type: Sub-task > Components: plugins >Reporter: Nigel Jones > Attachments: GaianDBAuth.docx > > > Applications/users could connect to gaianDB using their own authentication > information - for example userid/password in the simple case. Here the ranger > plugin will use that id for policy checks. > However in a multi tiered architecture a service id (aka non personal > account) may be used, and somehow the user to be impersonated is passed via > an additional property. This has a number of implications to the system > configuration, derby/gaiandb configuration & the plugin implementation. > Opening this Jira as a placeholder and will add a document soon (++days) on > the same to capture some of the discussion around this area in recent days. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (RANGER-1850) Impersonation/proxy user support for gaiandb ranger plugin
[ https://issues.apache.org/jira/browse/RANGER-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16312967#comment-16312967 ] Nigel Jones edited comment on RANGER-1850 at 1/5/18 11:16 AM: -- a) Point noted :-) b) Yes that is correct (I was aware of it, I should add to the docs), it's a current limitation caused by derby/us using backlevel. e) Correct we are not using this approach as this requires changes to applications and means derby sees one user (like 'gaiandb') whilst some the ranger plugin operates on another. It also means only sql statement execution uses this proxied user id, whilst jdbc metadata queries etc do not. Confusing and more divergent from base derby security & the way the derby security manager works. By using the proper derby plugin approach derby consistently sees the same userid as gaian regardless of the API call. We are also closer then to the intent of the derby community in terms of extension points, and can build on that approach as we update gaiandb in future f) The plugin is called when a connection is made from an application, via the jdbc driver, into gaiandb. It's right at the top.. It is not called per data source (that is something gaiandb manages using the auth details configured in gaiandb_config.properties for each source) was (Author: jonesn): a) Point accepted :-) > Impersonation/proxy user support for gaiandb ranger plugin > -- > > Key: RANGER-1850 > URL: https://issues.apache.org/jira/browse/RANGER-1850 > Project: Ranger > Issue Type: Sub-task > Components: plugins >Reporter: Nigel Jones > Attachments: GaianDBAuth.docx > > > Applications/users could connect to gaianDB using their own authentication > information - for example userid/password in the simple case. Here the ranger > plugin will use that id for policy checks. > However in a multi tiered architecture a service id (aka non personal > account) may be used, and somehow the user to be impersonated is passed via > an additional property. This has a number of implications to the system > configuration, derby/gaiandb configuration & the plugin implementation. > Opening this Jira as a placeholder and will add a document soon (++days) on > the same to capture some of the discussion around this area in recent days. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RANGER-1850) Impersonation/proxy user support for gaiandb ranger plugin
[ https://issues.apache.org/jira/browse/RANGER-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16312967#comment-16312967 ] Nigel Jones commented on RANGER-1850: - a) Point accepted :-) > Impersonation/proxy user support for gaiandb ranger plugin > -- > > Key: RANGER-1850 > URL: https://issues.apache.org/jira/browse/RANGER-1850 > Project: Ranger > Issue Type: Sub-task > Components: plugins >Reporter: Nigel Jones > Attachments: GaianDBAuth.docx > > > Applications/users could connect to gaianDB using their own authentication > information - for example userid/password in the simple case. Here the ranger > plugin will use that id for policy checks. > However in a multi tiered architecture a service id (aka non personal > account) may be used, and somehow the user to be impersonated is passed via > an additional property. This has a number of implications to the system > configuration, derby/gaiandb configuration & the plugin implementation. > Opening this Jira as a placeholder and will add a document soon (++days) on > the same to capture some of the discussion around this area in recent days. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RANGER-1850) Impersonation/proxy user support for gaiandb ranger plugin
[ https://issues.apache.org/jira/browse/RANGER-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16312899#comment-16312899 ] Nigel Jones commented on RANGER-1850: - Thanks for the feedback a) proxy auth fail -- I think both approaches make some sense. I'll think about which makes sense. Note that to some extent this is a shorter term fix since there's a lot more to do (IMO) for a full proxy support. for example I'd like to be able to grant specific permissions to the service account to permit it the proxy capability, and potentially vice versa to allow a user to be impersonated. This starts getting into the derby security manager in more detail, so should be based on using more up to date derby code (current) and working with the derby community. There's also here a tension between the implementation of the policy plugin and the security manager. We can't do that just yet so a 'sufficient' solution is all I'm going for at this point. b) create-schema well.. creates a schema for the user. I wouldn't have expected it to be needed either - since my queries are against fully qualified views like gaiandb.vcustomer -- but this fails with the older derby code gaiandb is using. From the mailing list (derby) it appears this is a bug as it only affects views and not tables.. but I can't move to current derby code without gaiandb rejigging. We've already started work to rebuild gaiandb and start updating libraries, but that is a longer term effort. So for now the pragmatic workaround is to add this flag. I did this in the plugin to reduce impact/complexity/odd errors from users of the code. Similar to a) it's a reflection of a minimal solution pending a much bigger change to bring gaiandb up to current code levels c) Which docs are you referring to? If gaiandb - correct. this is an additional plugin to be applied on top. In the future I hope we can make this a core capability, but this is a pragmatic approach on existing levels. If the docs for this plugin, agree that work isn't complete yet. Feel free to clarify what info you'll need and I'll add it in the README.md once the code moves into the ranger repo d) For the use case I'm addressing, auth against oracle is via the NPA defined for oracle (different to the gaiandb NPA), and those credentials are managed via the gaiandb configuration as is the case today. As far as gaiandb/derby is concerned the 'user' is the proxied user ie nigel, david etc as that is what the 'User' parm on connection is set to (one reason I used it with additional proxy-uid, proxy-password for the uid/password I wanted to use for the authentication check). So once the query leaves derby we are purely using the oracle NPA, and no attempt is made to 'pass down' the actual end user to the db. This would be very useful for audit, but the method to do this will vary by each db/source gaiandb supports. It would need more support in gaiandb itself for a framework, and then specifics for each supported db. Again we're up against the fact changes need to be made in this area in any case, so it didn't seem prudent to go down this route at this point. > Impersonation/proxy user support for gaiandb ranger plugin > -- > > Key: RANGER-1850 > URL: https://issues.apache.org/jira/browse/RANGER-1850 > Project: Ranger > Issue Type: Sub-task > Components: plugins >Reporter: Nigel Jones > Attachments: GaianDBAuth.docx > > > Applications/users could connect to gaianDB using their own authentication > information - for example userid/password in the simple case. Here the ranger > plugin will use that id for policy checks. > However in a multi tiered architecture a service id (aka non personal > account) may be used, and somehow the user to be impersonated is passed via > an additional property. This has a number of implications to the system > configuration, derby/gaiandb configuration & the plugin implementation. > Opening this Jira as a placeholder and will add a document soon (++days) on > the same to capture some of the discussion around this area in recent days. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RANGER-1850) Impersonation/proxy user support for gaiandb ranger plugin
[ https://issues.apache.org/jira/browse/RANGER-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16283345#comment-16283345 ] Nigel Jones commented on RANGER-1850: - I now have an initial version of this. Have added a proxy auth class to https://github.com/planetf1/gaiandb-policy-ranger/tree/master/src/main/java/org/apache/gaiandb/security This allows two additional parms to be passed on the URL - or in the properties object. proxy-user proxy-pwd These should be set to the NPA account - ie gaiandb/password. Then the EXISTING 'user' parm gets set to nigel, roger, ernie etc. A little glitch that password has to be non null for now - but is ignored. If proxy auth fails, or those parms aren't present, auth falls back to the existing gaiandb mechanism (which itself falls back to the native derby authenticator). By setting user to the 'real' user, the default schema is now 'jonesn' - which I wanted since I think the most secure approach is to allow all of the existing derby auth support to work as normal - the ONLY difference is really in how we connect.. so queries need to be fully qualified ie 'select * from gaiandb.vemployee' The proxy auth forces additional properties 'create=true' on the connection as otherwise derby will fail even with the fully qualified schema, if that schema does not exist. (at least in the older version used in gaiandb) Bottom line - can now authenticate using gaiandb user/password (an NPA), and passing in desired user - can select from gaiandb virtual tables - policy plugin gets invoked with correct user context - any other db controls respected using the real user (but this area needs very broad review post MVP) - can also connect as a regular user Need to figure out where to host this code. For now the update is in github as previously > Impersonation/proxy user support for gaiandb ranger plugin > -- > > Key: RANGER-1850 > URL: https://issues.apache.org/jira/browse/RANGER-1850 > Project: Ranger > Issue Type: Sub-task > Components: plugins >Reporter: Nigel Jones > Attachments: GaianDBAuth.docx > > > Applications/users could connect to gaianDB using their own authentication > information - for example userid/password in the simple case. Here the ranger > plugin will use that id for policy checks. > However in a multi tiered architecture a service id (aka non personal > account) may be used, and somehow the user to be impersonated is passed via > an additional property. This has a number of implications to the system > configuration, derby/gaiandb configuration & the plugin implementation. > Opening this Jira as a placeholder and will add a document soon (++days) on > the same to capture some of the discussion around this area in recent days. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (RANGER-1488) Create Ranger plugin for gaiandb
[ https://issues.apache.org/jira/browse/RANGER-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272877#comment-16272877 ] Nigel Jones edited comment on RANGER-1488 at 11/30/17 4:16 PM: --- Added some notes on what I've seen so far as I run through the gaianDB code. I will look at subtask 8 .. but this was needed as prep. Will continue next week The test data source used was https://cwiki.apache.org/confluence/display/ATLAS/Example+of+virtualizing+MySQL+employee+data+via+gaianDB - there's further gaiandb info via the atlas wiki was (Author: jonesn): Added some notes on what I've seen so far as I run through the gaianDB code. I will look at subtask 8 .. but this was needed as prep. Will continue next week > Create Ranger plugin for gaiandb > > > Key: RANGER-1488 > URL: https://issues.apache.org/jira/browse/RANGER-1488 > Project: Ranger > Issue Type: New Feature > Components: plugins >Reporter: Nigel Jones >Assignee: Nigel Jones > Labels: VirtualDataConnector > Attachments: GaianDebuggingResearch.docx > > > GaianDB is a distributed, federated database built on Apache Derby. > The documentation is at > https://github.com/gaiandb/gaiandb/blob/master/README.md - navigate up for > the source. > As part of a Virtual Connector Project based on Atlas we are using gaianDB to > provide a virtualization layer. We need to control access to underlying > resources and will be building a Ranger plugin for gaiandb to support this. > GaianDB already has support for a form of policy plugin which allows the SQL > to be intercepted. > It is unclear if this code will sit > - external to ranger/atlas ie in the gaianDB sources > - as a sample in the Atlas project (so that we collate what is needed for > the virtual connector project) > - as a sample, or component in the Ranger project > However it seems helpful to describe the proposal here and get > feedback/dialogue in the ranger community :-) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RANGER-1488) Create Ranger plugin for gaiandb
[ https://issues.apache.org/jira/browse/RANGER-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272877#comment-16272877 ] Nigel Jones commented on RANGER-1488: - Added some notes on what I've seen so far as I run through the gaianDB code. I will look at subtask 8 .. but this was needed as prep. Will continue next week > Create Ranger plugin for gaiandb > > > Key: RANGER-1488 > URL: https://issues.apache.org/jira/browse/RANGER-1488 > Project: Ranger > Issue Type: New Feature > Components: plugins >Reporter: Nigel Jones >Assignee: Nigel Jones > Labels: VirtualDataConnector > Attachments: GaianDebuggingResearch.docx > > > GaianDB is a distributed, federated database built on Apache Derby. > The documentation is at > https://github.com/gaiandb/gaiandb/blob/master/README.md - navigate up for > the source. > As part of a Virtual Connector Project based on Atlas we are using gaianDB to > provide a virtualization layer. We need to control access to underlying > resources and will be building a Ranger plugin for gaiandb to support this. > GaianDB already has support for a form of policy plugin which allows the SQL > to be intercepted. > It is unclear if this code will sit > - external to ranger/atlas ie in the gaianDB sources > - as a sample in the Atlas project (so that we collate what is needed for > the virtual connector project) > - as a sample, or component in the Ranger project > However it seems helpful to describe the proposal here and get > feedback/dialogue in the ranger community :-) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (RANGER-1488) Create Ranger plugin for gaiandb
[ https://issues.apache.org/jira/browse/RANGER-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated RANGER-1488: Attachment: GaianDebuggingResearch.docx Some rough notes of working through the gaiandb code > Create Ranger plugin for gaiandb > > > Key: RANGER-1488 > URL: https://issues.apache.org/jira/browse/RANGER-1488 > Project: Ranger > Issue Type: New Feature > Components: plugins >Reporter: Nigel Jones >Assignee: Nigel Jones > Labels: VirtualDataConnector > Attachments: GaianDebuggingResearch.docx > > > GaianDB is a distributed, federated database built on Apache Derby. > The documentation is at > https://github.com/gaiandb/gaiandb/blob/master/README.md - navigate up for > the source. > As part of a Virtual Connector Project based on Atlas we are using gaianDB to > provide a virtualization layer. We need to control access to underlying > resources and will be building a Ranger plugin for gaiandb to support this. > GaianDB already has support for a form of policy plugin which allows the SQL > to be intercepted. > It is unclear if this code will sit > - external to ranger/atlas ie in the gaianDB sources > - as a sample in the Atlas project (so that we collate what is needed for > the virtual connector project) > - as a sample, or component in the Ranger project > However it seems helpful to describe the proposal here and get > feedback/dialogue in the ranger community :-) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (RANGER-1908) Build (unit tests) failed in HBase Security Plugin
[ https://issues.apache.org/jira/browse/RANGER-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones reassigned RANGER-1908: --- Assignee: Nigel Jones > Build (unit tests) failed in HBase Security Plugin > -- > > Key: RANGER-1908 > URL: https://issues.apache.org/jira/browse/RANGER-1908 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Nigel Jones >Assignee: Nigel Jones > Attachments: ranger-20171124a.log > > > MacOS > Java 1.8 (152, oracle) > Maven 3.5.0 > Build fails at > [INFO] HBase Security Plugin .. FAILURE [11:36 > min] > The failing test is: > [INFO] Running > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest > Formatting using clusterid: testClusterID > [ERROR] Tests run: 20, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: > 689.962 s <<< FAILURE! - in > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest > [ERROR] > testTagBasedTablePolicy(org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest) > Time elapsed: 611.879 s <<< ERROR! > org.apache.hadoop.hbase.exceptions.TimeoutIOException: > java.util.concurrent.TimeoutException: The procedure 18 is still running > at > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest.testTagBasedTablePolicy(HBaseRangerAuthorizationTest.java:780) > Caused by: java.util.concurrent.TimeoutException: The procedure 18 is still > running > at > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest.testTagBasedTablePolicy(HBaseRangerAuthorizationTest.java:780) > The next part of the test is ok: > [INFO] Running > org.apache.ranger.authorization.hbase.RangerAuthorizationFilterTest > [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.505 > s - in org.apache.ranger.authorization.hbase.RangerAuthorizationFilterTest -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (RANGER-1908) Build (unit tests) failed in HBase Security Plugin
[ https://issues.apache.org/jira/browse/RANGER-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones resolved RANGER-1908. - Resolution: Cannot Reproduce > Build (unit tests) failed in HBase Security Plugin > -- > > Key: RANGER-1908 > URL: https://issues.apache.org/jira/browse/RANGER-1908 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Nigel Jones >Assignee: Nigel Jones > Attachments: ranger-20171124a.log > > > MacOS > Java 1.8 (152, oracle) > Maven 3.5.0 > Build fails at > [INFO] HBase Security Plugin .. FAILURE [11:36 > min] > The failing test is: > [INFO] Running > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest > Formatting using clusterid: testClusterID > [ERROR] Tests run: 20, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: > 689.962 s <<< FAILURE! - in > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest > [ERROR] > testTagBasedTablePolicy(org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest) > Time elapsed: 611.879 s <<< ERROR! > org.apache.hadoop.hbase.exceptions.TimeoutIOException: > java.util.concurrent.TimeoutException: The procedure 18 is still running > at > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest.testTagBasedTablePolicy(HBaseRangerAuthorizationTest.java:780) > Caused by: java.util.concurrent.TimeoutException: The procedure 18 is still > running > at > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest.testTagBasedTablePolicy(HBaseRangerAuthorizationTest.java:780) > The next part of the test is ok: > [INFO] Running > org.apache.ranger.authorization.hbase.RangerAuthorizationFilterTest > [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.505 > s - in org.apache.ranger.authorization.hbase.RangerAuthorizationFilterTest -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (RANGER-1908) Build (unit tests) failed in HBase Security Plugin
[ https://issues.apache.org/jira/browse/RANGER-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266840#comment-16266840 ] Nigel Jones edited comment on RANGER-1908 at 11/27/17 2:00 PM: --- More tests * MacOS - 4 consecutive builds all clean * VM (a) @IBM. Failed 2/3, worked 1/3. looks timing related unit test case failure (testing Kafka security plugin -- not the HBase one) * VM (b) @Azure (lower spec) failed 3/3 same as above. timing related At this point I will close this JIRA as non reproducible after around 8 builds Will keep an eye on any timing related issues to see if perhaps we need to relax some timings and raise a new JIRA if so. was (Author: jonesn): More tests * MacOS - 4 consecutive builds all clean * VM (a) @IBM. Failed 2/3, worked 1/3. looks timing related unit test case failure * VM (b) @Azure (lower spec) failed 3/3 same as above. timing related At this point I will close this JIRA as non reproducible. Will keep an eye on any timing related issues to see if perhaps we need to relax some timings and raise a new JIRA if so. > Build (unit tests) failed in HBase Security Plugin > -- > > Key: RANGER-1908 > URL: https://issues.apache.org/jira/browse/RANGER-1908 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Nigel Jones > Attachments: ranger-20171124a.log > > > MacOS > Java 1.8 (152, oracle) > Maven 3.5.0 > Build fails at > [INFO] HBase Security Plugin .. FAILURE [11:36 > min] > The failing test is: > [INFO] Running > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest > Formatting using clusterid: testClusterID > [ERROR] Tests run: 20, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: > 689.962 s <<< FAILURE! - in > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest > [ERROR] > testTagBasedTablePolicy(org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest) > Time elapsed: 611.879 s <<< ERROR! > org.apache.hadoop.hbase.exceptions.TimeoutIOException: > java.util.concurrent.TimeoutException: The procedure 18 is still running > at > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest.testTagBasedTablePolicy(HBaseRangerAuthorizationTest.java:780) > Caused by: java.util.concurrent.TimeoutException: The procedure 18 is still > running > at > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest.testTagBasedTablePolicy(HBaseRangerAuthorizationTest.java:780) > The next part of the test is ok: > [INFO] Running > org.apache.ranger.authorization.hbase.RangerAuthorizationFilterTest > [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.505 > s - in org.apache.ranger.authorization.hbase.RangerAuthorizationFilterTest -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RANGER-1908) Build (unit tests) failed in HBase Security Plugin
[ https://issues.apache.org/jira/browse/RANGER-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266840#comment-16266840 ] Nigel Jones commented on RANGER-1908: - More tests * MacOS - 4 consecutive builds all clean * VM (a) @IBM. Failed 2/3, worked 1/3. looks timing related unit test case failure * VM (b) @Azure (lower spec) failed 3/3 same as above. timing related At this point I will close this JIRA as non reproducible. Will keep an eye on any timing related issues to see if perhaps we need to relax some timings and raise a new JIRA if so. > Build (unit tests) failed in HBase Security Plugin > -- > > Key: RANGER-1908 > URL: https://issues.apache.org/jira/browse/RANGER-1908 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Nigel Jones > Attachments: ranger-20171124a.log > > > MacOS > Java 1.8 (152, oracle) > Maven 3.5.0 > Build fails at > [INFO] HBase Security Plugin .. FAILURE [11:36 > min] > The failing test is: > [INFO] Running > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest > Formatting using clusterid: testClusterID > [ERROR] Tests run: 20, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: > 689.962 s <<< FAILURE! - in > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest > [ERROR] > testTagBasedTablePolicy(org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest) > Time elapsed: 611.879 s <<< ERROR! > org.apache.hadoop.hbase.exceptions.TimeoutIOException: > java.util.concurrent.TimeoutException: The procedure 18 is still running > at > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest.testTagBasedTablePolicy(HBaseRangerAuthorizationTest.java:780) > Caused by: java.util.concurrent.TimeoutException: The procedure 18 is still > running > at > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest.testTagBasedTablePolicy(HBaseRangerAuthorizationTest.java:780) > The next part of the test is ok: > [INFO] Running > org.apache.ranger.authorization.hbase.RangerAuthorizationFilterTest > [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.505 > s - in org.apache.ranger.authorization.hbase.RangerAuthorizationFilterTest -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (RANGER-1908) Build (unit tests) failed in HBase Security Plugin
[ https://issues.apache.org/jira/browse/RANGER-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated RANGER-1908: Summary: Build (unit tests) failed in HBase Security Plugin (was: Build failed in HBase Security Plugin) > Build (unit tests) failed in HBase Security Plugin > -- > > Key: RANGER-1908 > URL: https://issues.apache.org/jira/browse/RANGER-1908 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Nigel Jones > Attachments: ranger-20171124a.log > > > MacOS > Java 1.8 (152, oracle) > Maven 3.5.0 > Build fails at > [INFO] HBase Security Plugin .. FAILURE [11:36 > min] > The failing test is: > [INFO] Running > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest > Formatting using clusterid: testClusterID > [ERROR] Tests run: 20, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: > 689.962 s <<< FAILURE! - in > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest > [ERROR] > testTagBasedTablePolicy(org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest) > Time elapsed: 611.879 s <<< ERROR! > org.apache.hadoop.hbase.exceptions.TimeoutIOException: > java.util.concurrent.TimeoutException: The procedure 18 is still running > at > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest.testTagBasedTablePolicy(HBaseRangerAuthorizationTest.java:780) > Caused by: java.util.concurrent.TimeoutException: The procedure 18 is still > running > at > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest.testTagBasedTablePolicy(HBaseRangerAuthorizationTest.java:780) > The next part of the test is ok: > [INFO] Running > org.apache.ranger.authorization.hbase.RangerAuthorizationFilterTest > [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.505 > s - in org.apache.ranger.authorization.hbase.RangerAuthorizationFilterTest -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RANGER-1909) Build failure in Kafka Security plugin Unit Tests
[ https://issues.apache.org/jira/browse/RANGER-1909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16265425#comment-16265425 ] Nigel Jones commented on RANGER-1909: - This is on Ubuntu 17.10 (azure) Another linux environment with 16.04 did build the same source fine, though macos failed (see RANGER-1908) This could be timing related, and down to the fact I'm using a 'burstable' vm within a small budget (class B2S - 2 CPU. but min cap 20%) This was initially used to compare against the system that failed with 1908 (which is a fast macbook pro), but subsequently I used a non-restricted 4 CPU environment which does build cleanly. Opening jira to identify/record cause > Build failure in Kafka Security plugin Unit Tests > - > > Key: RANGER-1909 > URL: https://issues.apache.org/jira/browse/RANGER-1909 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Nigel Jones > Attachments: ranger-20171124a.log > > > Java 1.8 (151, openjdk) > Maven 3.5.0 > Build fails at > [INFO] KAFKA Security Plugin .. FAILURE [22:07 > min] > [INFO] Running > org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerGSSTest > log4j:WARN No appenders could be found for logger > (org.apache.kerby.kerberos.kerb.identity.backend.AbstractIdentityBackend). > log4j:WARN Please initialize the log4j system properly. > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more > info. > [ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: > 1,120.51 s <<< FAILURE! - in > org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerGSSTest > [ERROR] > testAuthorizedRead(org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerGSSTest) > Time elapsed: 1,109.032 s <<< FAILURE! > java.lang.AssertionError > at > org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerGSSTest.testAuthorizedRead(KafkaRangerAuthorizerGSSTest.java:243) > [INFO] Running > org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerTest > log4j:WARN No appenders could be found for logger > (org.apache.zookeeper.server.ZooKeeperServerMain). > log4j:WARN Please initialize the log4j system properly. > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more > info. > [INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: > 114.827 s - in > org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerTest > [INFO] Running > org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerSASLSSLTest > log4j:WARN No appenders could be found for logger > (org.apache.zookeeper.server.ZooKeeperServerMain). > log4j:WARN Please initialize the log4j system properly. > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more > info. > [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 72.214 > s - in > org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerSASLSSLTest > [INFO] > [INFO] Results: > [INFO] > [ERROR] Failures: > [ERROR] KafkaRangerAuthorizerGSSTest.testAuthorizedRead:243 > [INFO] > [ERROR] Tests run: 8, Failures: 1, Errors: 0, Skipped: 0 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (RANGER-1909) Build failure in Kafka Security plugin Unit Tests
[ https://issues.apache.org/jira/browse/RANGER-1909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated RANGER-1909: Attachment: ranger-20171124a.log > Build failure in Kafka Security plugin Unit Tests > - > > Key: RANGER-1909 > URL: https://issues.apache.org/jira/browse/RANGER-1909 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Nigel Jones > Attachments: ranger-20171124a.log > > > Java 1.8 (151, openjdk) > Maven 3.5.0 > Build fails at > [INFO] KAFKA Security Plugin .. FAILURE [22:07 > min] > [INFO] Running > org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerGSSTest > log4j:WARN No appenders could be found for logger > (org.apache.kerby.kerberos.kerb.identity.backend.AbstractIdentityBackend). > log4j:WARN Please initialize the log4j system properly. > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more > info. > [ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: > 1,120.51 s <<< FAILURE! - in > org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerGSSTest > [ERROR] > testAuthorizedRead(org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerGSSTest) > Time elapsed: 1,109.032 s <<< FAILURE! > java.lang.AssertionError > at > org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerGSSTest.testAuthorizedRead(KafkaRangerAuthorizerGSSTest.java:243) > [INFO] Running > org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerTest > log4j:WARN No appenders could be found for logger > (org.apache.zookeeper.server.ZooKeeperServerMain). > log4j:WARN Please initialize the log4j system properly. > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more > info. > [INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: > 114.827 s - in > org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerTest > [INFO] Running > org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerSASLSSLTest > log4j:WARN No appenders could be found for logger > (org.apache.zookeeper.server.ZooKeeperServerMain). > log4j:WARN Please initialize the log4j system properly. > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more > info. > [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 72.214 > s - in > org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerSASLSSLTest > [INFO] > [INFO] Results: > [INFO] > [ERROR] Failures: > [ERROR] KafkaRangerAuthorizerGSSTest.testAuthorizedRead:243 > [INFO] > [ERROR] Tests run: 8, Failures: 1, Errors: 0, Skipped: 0 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (RANGER-1909) Build failure in Kafka Security plugin Unit Tests
Nigel Jones created RANGER-1909: --- Summary: Build failure in Kafka Security plugin Unit Tests Key: RANGER-1909 URL: https://issues.apache.org/jira/browse/RANGER-1909 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Nigel Jones Java 1.8 (151, openjdk) Maven 3.5.0 Build fails at [INFO] KAFKA Security Plugin .. FAILURE [22:07 min] [INFO] Running org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerGSSTest log4j:WARN No appenders could be found for logger (org.apache.kerby.kerberos.kerb.identity.backend.AbstractIdentityBackend). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. [ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1,120.51 s <<< FAILURE! - in org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerGSSTest [ERROR] testAuthorizedRead(org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerGSSTest) Time elapsed: 1,109.032 s <<< FAILURE! java.lang.AssertionError at org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerGSSTest.testAuthorizedRead(KafkaRangerAuthorizerGSSTest.java:243) [INFO] Running org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerTest log4j:WARN No appenders could be found for logger (org.apache.zookeeper.server.ZooKeeperServerMain). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. [INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 114.827 s - in org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerTest [INFO] Running org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerSASLSSLTest log4j:WARN No appenders could be found for logger (org.apache.zookeeper.server.ZooKeeperServerMain). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 72.214 s - in org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerSASLSSLTest [INFO] [INFO] Results: [INFO] [ERROR] Failures: [ERROR] KafkaRangerAuthorizerGSSTest.testAuthorizedRead:243 [INFO] [ERROR] Tests run: 8, Failures: 1, Errors: 0, Skipped: 0 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (RANGER-1908) Build failed in HBase Security Plugin
[ https://issues.apache.org/jira/browse/RANGER-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated RANGER-1908: Attachment: ranger-20171124a.log > Build failed in HBase Security Plugin > - > > Key: RANGER-1908 > URL: https://issues.apache.org/jira/browse/RANGER-1908 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Nigel Jones > Attachments: ranger-20171124a.log > > > MacOS > Java 1.8 (152, oracle) > Maven 3.5.0 > Build fails at > [INFO] HBase Security Plugin .. FAILURE [11:36 > min] > The failing test is: > [INFO] Running > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest > Formatting using clusterid: testClusterID > [ERROR] Tests run: 20, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: > 689.962 s <<< FAILURE! - in > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest > [ERROR] > testTagBasedTablePolicy(org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest) > Time elapsed: 611.879 s <<< ERROR! > org.apache.hadoop.hbase.exceptions.TimeoutIOException: > java.util.concurrent.TimeoutException: The procedure 18 is still running > at > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest.testTagBasedTablePolicy(HBaseRangerAuthorizationTest.java:780) > Caused by: java.util.concurrent.TimeoutException: The procedure 18 is still > running > at > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest.testTagBasedTablePolicy(HBaseRangerAuthorizationTest.java:780) > The next part of the test is ok: > [INFO] Running > org.apache.ranger.authorization.hbase.RangerAuthorizationFilterTest > [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.505 > s - in org.apache.ranger.authorization.hbase.RangerAuthorizationFilterTest -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (RANGER-1908) Build failed in HBase Security Plugin
Nigel Jones created RANGER-1908: --- Summary: Build failed in HBase Security Plugin Key: RANGER-1908 URL: https://issues.apache.org/jira/browse/RANGER-1908 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Nigel Jones MacOS Java 1.8 (152, oracle) Maven 3.5.0 Build fails at [INFO] HBase Security Plugin .. FAILURE [11:36 min] The failing test is: [INFO] Running org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest Formatting using clusterid: testClusterID [ERROR] Tests run: 20, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 689.962 s <<< FAILURE! - in org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest [ERROR] testTagBasedTablePolicy(org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest) Time elapsed: 611.879 s <<< ERROR! org.apache.hadoop.hbase.exceptions.TimeoutIOException: java.util.concurrent.TimeoutException: The procedure 18 is still running at org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest.testTagBasedTablePolicy(HBaseRangerAuthorizationTest.java:780) Caused by: java.util.concurrent.TimeoutException: The procedure 18 is still running at org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest.testTagBasedTablePolicy(HBaseRangerAuthorizationTest.java:780) The next part of the test is ok: [INFO] Running org.apache.ranger.authorization.hbase.RangerAuthorizationFilterTest [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.505 s - in org.apache.ranger.authorization.hbase.RangerAuthorizationFilterTest -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (RANGER-1850) Impersonation/proxy user support for gaiandb ranger plugin
[ https://issues.apache.org/jira/browse/RANGER-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated RANGER-1850: Attachment: GaianDBAuth.docx very initial draft. For reference. Will add more info here once ready for review > Impersonation/proxy user support for gaiandb ranger plugin > -- > > Key: RANGER-1850 > URL: https://issues.apache.org/jira/browse/RANGER-1850 > Project: Ranger > Issue Type: Sub-task > Components: plugins >Reporter: Nigel Jones > Attachments: GaianDBAuth.docx > > > Applications/users could connect to gaianDB using their own authentication > information - for example userid/password in the simple case. Here the ranger > plugin will use that id for policy checks. > However in a multi tiered architecture a service id (aka non personal > account) may be used, and somehow the user to be impersonated is passed via > an additional property. This has a number of implications to the system > configuration, derby/gaiandb configuration & the plugin implementation. > Opening this Jira as a placeholder and will add a document soon (++days) on > the same to capture some of the discussion around this area in recent days. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RANGER-1488) Create Ranger plugin for gaiandb
[ https://issues.apache.org/jira/browse/RANGER-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16157125#comment-16157125 ] Nigel Jones commented on RANGER-1488: - Originally it was planned that authorization would be based purely on the user that connects to gaianDB (which can support local users, or LDAP - the latter being more likely in an enterprise) ie this user would be used within the ranger/gaiandb plugin to determine access priviliges. However feedback this week has suggested that *in addition*, it should be possible to pass the userid as a connection parameter (derby supports this), since it will be a 'non personal' or 'generic' server user id that may be connecting to gaiandb in some use cases. > Create Ranger plugin for gaiandb > > > Key: RANGER-1488 > URL: https://issues.apache.org/jira/browse/RANGER-1488 > Project: Ranger > Issue Type: New Feature > Components: plugins >Reporter: Nigel Jones >Assignee: Nigel Jones > Labels: VirtualDataConnector > > GaianDB is a distributed, federated database built on Apache Derby. > The documentation is at > https://github.com/gaiandb/gaiandb/blob/master/README.md - navigate up for > the source. > As part of a Virtual Connector Project based on Atlas we are using gaianDB to > provide a virtualization layer. We need to control access to underlying > resources and will be building a Ranger plugin for gaiandb to support this. > GaianDB already has support for a form of policy plugin which allows the SQL > to be intercepted. > It is unclear if this code will sit > - external to ranger/atlas ie in the gaianDB sources > - as a sample in the Atlas project (so that we collate what is needed for > the virtual connector project) > - as a sample, or component in the Ranger project > However it seems helpful to describe the proposal here and get > feedback/dialogue in the ranger community :-) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RANGER-1488) Create Ranger plugin for gaiandb
[ https://issues.apache.org/jira/browse/RANGER-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16157116#comment-16157116 ] Nigel Jones commented on RANGER-1488: - An outline of the broader use cases within which the gaiandb ranger plugin would be used can be found at https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=69407333 However the plugin should be self-contained, and not dependent on the other ongoing work around atlas and open metadata. It should fundamentally work within the existing ranger framework, and not be dependent on the ongoing work in particular in atlas around open metadata, OMAS interfaces and any proposed changes to tagsync, metadata exchange etc. the capabilities will include - simple permit/deny on data access - masking ( is sufficient as a first step, but could be elaborate as in hive) - supporting both resource based policies & tag based policies Not required by current use cases but a useful future enhancement would include - row filtering A desire has also been expressed such that if a user were to do a 'select *' from a gaiandb virtual table, any columns the user cannot see should be stripped out. However this may cause inconsistencies with any metadata queries (which cannot be filtered by a ranger plugin since gaiandb does not yet support this). A consideration, but likely to be tricky at this time. > Create Ranger plugin for gaiandb > > > Key: RANGER-1488 > URL: https://issues.apache.org/jira/browse/RANGER-1488 > Project: Ranger > Issue Type: New Feature > Components: plugins >Reporter: Nigel Jones >Assignee: Nigel Jones > Labels: VirtualDataConnector > > GaianDB is a distributed, federated database built on Apache Derby. > The documentation is at > https://github.com/gaiandb/gaiandb/blob/master/README.md - navigate up for > the source. > As part of a Virtual Connector Project based on Atlas we are using gaianDB to > provide a virtualization layer. We need to control access to underlying > resources and will be building a Ranger plugin for gaiandb to support this. > GaianDB already has support for a form of policy plugin which allows the SQL > to be intercepted. > It is unclear if this code will sit > - external to ranger/atlas ie in the gaianDB sources > - as a sample in the Atlas project (so that we collate what is needed for > the virtual connector project) > - as a sample, or component in the Ranger project > However it seems helpful to describe the proposal here and get > feedback/dialogue in the ranger community :-) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RANGER-1572) Provide simple example of running ranger with docker
[ https://issues.apache.org/jira/browse/RANGER-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150756#comment-16150756 ] Nigel Jones commented on RANGER-1572: - In ATLAS-2012 we added a docker script to Atlas, this was quite different to my github example above. Ranger however is a little more complex in that beyond understanding the actual component, it's more difficult than atlas to get value from as a standalone component. In Atlas discussions we've acknowledged this and raised ATLAS-2095 though its had little detailed thought yet. > Provide simple example of running ranger with docker > > > Key: RANGER-1572 > URL: https://issues.apache.org/jira/browse/RANGER-1572 > Project: Ranger > Issue Type: New Feature > Components: Ranger >Reporter: Nigel Jones >Assignee: Nigel Jones > > When experimenting with Ranger it's very useful to be able to quickly run > ranger admin/solr/mysql > Since I needed this for my education/experimentation I thought it might be > useful to others. Work in progress and doesn't work yet but raising this JIRA > in case anyone else has done the same, or is interested ;-) > For now the code (just a few scripts) is at > https://github.com/planetf1/ranger-docker whilst I try and get this working. > Perhaps this could be added as an example down the line, or even used within > the build to create an image -- though there's lots of variables in terms of > config. > As above.. not working yet ;-) WIP! -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RANGER-1486) New usersync alternative for Atlas (vdc)
[ https://issues.apache.org/jira/browse/RANGER-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150711#comment-16150711 ] Nigel Jones commented on RANGER-1486: - Thanks... the suggestion is purely to scope the groups based on a list from Atlas (we'll call them roles there as that usually how they are used). That's really the only change, other than effectively an additional predicate, usersync would work in the same way. I'm hoping it's a pragmatic way to significantly reduce how much user/group info is retrieved from ldap in a large enterprise environment with many many different apps & environments, and within which currently only a few roles are relevant to the data lake environment (& hence atlas/ranger). Maybe in future there's other ideas, including whether ranger even needs to store these (I know it does now..) but that's for another day Makes sense? Thanks, Nigel. > New usersync alternative for Atlas (vdc) > > > Key: RANGER-1486 > URL: https://issues.apache.org/jira/browse/RANGER-1486 > Project: Ranger > Issue Type: New Feature > Components: usersync >Reporter: Nigel Jones >Assignee: Nigel Jones > Labels: VirtualDataConnector > > As part of the Atlas Virtualization Data Connector work we are using this > within a large enterprise with a lot of users & groups stored in ldap. > The connector -- which has a ranger plugin to apply access control policies > -- is used by a relatively small subset of these users. However that can't > easily be transcribed to an optimal ldap query. > Since Atlas will have the definitive list of roles that are being used, this > new usersync will instead retrieve a list of roles from Atlas, and will then > use this list to retrieve only those users found in this list of roles from > LDAP. > This is an alternative usersync so shouldn't conflict and will use the same > ranger APIs -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RANGER-1454) Support of Atlas GAF OMAS
[ https://issues.apache.org/jira/browse/RANGER-1454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150546#comment-16150546 ] Nigel Jones commented on RANGER-1454: - The assumption in this Jira is that the existing Ranger REST API remains as it is today. These changes to support new Atlas APIs are initially focussed only at getting tagsync working properly with the new data model & approach to classifications. Once in place, since the Atlas Governance Engine OMAS api is consumer centric, it is intended purely for enforcement engines such as ranger, and therefore the Atlas can can extend as needed to improve integration with ranger. Better fine grained notifications from atlas to ranger may also make it easier to update policies more quickly, but would require other changes in ranger. > Support of Atlas GAF OMAS > -- > > Key: RANGER-1454 > URL: https://issues.apache.org/jira/browse/RANGER-1454 > Project: Ranger > Issue Type: New Feature > Components: tagsync >Reporter: Nigel Jones >Assignee: Nigel Jones > Labels: VirtualDataConnector > > A new v2 glossary capability is proposed for Atlas in ATLAS-1410. > As part of this the glossary model becomes more sophisticated. In order to > preserve the current simple tag(trait type):parms(trait instance)-entity > relationship a new tagsync process will be developed that makes use of a new > API proposed in ATLAS-1662. This would be an alternative to the current one > so that existing users could continue unaffected, and a change is only > required if moving to the v2 glossary implementation > This will also allow the process of retrieving tags to be more efficient, and > this new API can form a consumer-centric interface to support multiple > enforcement technologies, including, but not restricted to, Ranger. > feel free to assign to me/modify permissions ;-) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (RANGER-1758) Atlas GE OMAS : Support new Kafka notifications
Nigel Jones created RANGER-1758: --- Summary: Atlas GE OMAS : Support new Kafka notifications Key: RANGER-1758 URL: https://issues.apache.org/jira/browse/RANGER-1758 Project: Ranger Issue Type: Sub-task Components: tagsync Reporter: Nigel Jones This JIRA adds in support for the new Governance Engine OMAS notifications coming from Atlas, avoiding the need to pull the entire list of classifications/assets from Atlas anytime there's an update, or periodically. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (RANGER-1757) Atlas GE OMAS : Support new REST api
Nigel Jones created RANGER-1757: --- Summary: Atlas GE OMAS : Support new REST api Key: RANGER-1757 URL: https://issues.apache.org/jira/browse/RANGER-1757 Project: Ranger Issue Type: Sub-task Components: tagsync Reporter: Nigel Jones This subtask will be used to implement the initial REST api queries, which could also be repeated at intervals. I am splitting out this from the messaging/kafka notifications as the dependencies in atlas differ, and to aid in what can be implemented independently. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RANGER-1670) Change in Atlas Kafka consumer interface for Atlas tag sync.
[ https://issues.apache.org/jira/browse/RANGER-1670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150204#comment-16150204 ] Nigel Jones commented on RANGER-1670: - Hi, I thought it worth mentioning that beyond the just-being-released 0.8.1, there's a fair bit of work going in with Atlas currently, to enrichen the data model, and to provide new consumer centric interfaces. The glossary is being expanded a lot, to enhance the relationships between assets, business terms & classifications. The idea is to surface a more consumer centric view of this for ranger (and other enforcement engines). In RANGER-1454 I propose to switch to using this new API, but it's fair to say the focus so far has been on the atlas side, and it's only now we're getting more into the ranger aspects (thinking of!), which was a little tricky before the atlas side started firming up. I'd love your feedback on the ideas and happy to chat more. > Change in Atlas Kafka consumer interface for Atlas tag sync. > > > Key: RANGER-1670 > URL: https://issues.apache.org/jira/browse/RANGER-1670 > Project: Ranger > Issue Type: Bug > Components: tagsync >Reporter: Nixon Rodrigues > Attachments: RANGER-1670.patch > > > This change is dependent on upcoming 0.8.1 Atlas release. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RANGER-1486) New usersync alternative for Atlas (vdc)
[ https://issues.apache.org/jira/browse/RANGER-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147516#comment-16147516 ] Nigel Jones commented on RANGER-1486: - We don't yet have a concept of user roles in the proposed atlas model. I added thoughts in ATLAS-1768 though the specific dependency for usersync would be the API being delivered via ATLAS-1796 which can be dummied up prior to the necessary model updates. One further thought... to limit interdependencies between atlas and ranger I think we should fall back to the old behaviour if the atlas being contacted does not support the new API. Initially it might also be wise to have a property to enable the new behaviour (so as to not hit an existing/old atlas server with a new API request), or to figure out (new JIRA?) an appropriate more generic version check or capability strategy > New usersync alternative for Atlas (vdc) > > > Key: RANGER-1486 > URL: https://issues.apache.org/jira/browse/RANGER-1486 > Project: Ranger > Issue Type: New Feature > Components: usersync >Reporter: Nigel Jones >Assignee: Nigel Jones > Labels: VirtualDataConnector > > As part of the Atlas Virtualization Data Connector work we are using this > within a large enterprise with a lot of users & groups stored in ldap. > The connector -- which has a ranger plugin to apply access control policies > -- is used by a relatively small subset of these users. However that can't > easily be transcribed to an optimal ldap query. > Since Atlas will have the definitive list of roles that are being used, this > new usersync will instead retrieve a list of roles from Atlas, and will then > use this list to retrieve only those users found in this list of roles from > LDAP. > This is an alternative usersync so shouldn't conflict and will use the same > ranger APIs -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RANGER-1486) New usersync alternative for Atlas (vdc)
[ https://issues.apache.org/jira/browse/RANGER-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147417#comment-16147417 ] Nigel Jones commented on RANGER-1486: - * Making the use of restricted Atlas roles configurable makes sense - I'd add that * The role names wouldn't come from configuration - they would come from atlas via the 'gaf omas' call (note: this OMAS we are now calling the governance engine OMAS) (marketing, hr, datascientist etc). They would match the role names defined in LDAP. * I would apply the ldap filter too - indeed in some ways the list of roles from atlas forms an additional predicate. However I'm not a specialist on ldap queries. We do need to consider efficiency * We can get a notification (using the governance engine omas, which will deliver it over Kafka) when a new role is added or removed from atlas, which should cause us to redo the query. However new users we won't see as they are not defined in Atlas, only in LDAP & in general ldap doesn't offer an event. In fact I think with both OpenLDAP & Active Directory it would be possible to create triggers & generate notifications that ranger/usersync could use it's not really standard. Perhaps we should open a separate JIRA on that as it sounds like it could be genuinely useful in an enterprise environment (part B though would be then having a trigger mechanism to push the update not just to the ranger server, but to the plugins, and that's complex since they can be based on many technologies. At best an optional feature for a plugin?) * In terms of a role changing, currently in Atlas I don't think we're thinking of any more than the role name. so there's not really anything to change. Whilst ideally we should track the guid of the Atlas defined role, since this doesn't tie up with ldap in any way it's difficult to see we could get much value from this. I think in general the idea of being able to further restrict an ldap search makes sense in a large environment otherwise we're pointlessly pushing far too much user/group info into ranger, but we do need consensus on whether the community is happy that the scoping could come from Atlas. I hope that as long as the feature is optional this would be ok. > New usersync alternative for Atlas (vdc) > > > Key: RANGER-1486 > URL: https://issues.apache.org/jira/browse/RANGER-1486 > Project: Ranger > Issue Type: New Feature > Components: usersync >Reporter: Nigel Jones >Assignee: Nigel Jones > Labels: VirtualDataConnector > > As part of the Atlas Virtualization Data Connector work we are using this > within a large enterprise with a lot of users & groups stored in ldap. > The connector -- which has a ranger plugin to apply access control policies > -- is used by a relatively small subset of these users. However that can't > easily be transcribed to an optimal ldap query. > Since Atlas will have the definitive list of roles that are being used, this > new usersync will instead retrieve a list of roles from Atlas, and will then > use this list to retrieve only those users found in this list of roles from > LDAP. > This is an alternative usersync so shouldn't conflict and will use the same > ranger APIs -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (RANGER-1572) Provide simple example of running ranger with docker
[ https://issues.apache.org/jira/browse/RANGER-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated RANGER-1572: Issue Type: New Feature (was: Bug) > Provide simple example of running ranger with docker > > > Key: RANGER-1572 > URL: https://issues.apache.org/jira/browse/RANGER-1572 > Project: Ranger > Issue Type: New Feature > Components: Ranger >Reporter: Nigel Jones >Assignee: Nigel Jones > > When experimenting with Ranger it's very useful to be able to quickly run > ranger admin/solr/mysql > Since I needed this for my education/experimentation I thought it might be > useful to others. Work in progress and doesn't work yet but raising this JIRA > in case anyone else has done the same, or is interested ;-) > For now the code (just a few scripts) is at > https://github.com/planetf1/ranger-docker whilst I try and get this working. > Perhaps this could be added as an example down the line, or even used within > the build to create an image -- though there's lots of variables in terms of > config. > As above.. not working yet ;-) WIP! -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RANGER-676) Audit logging to Elasticsearch
[ https://issues.apache.org/jira/browse/RANGER-676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109098#comment-16109098 ] Nigel Jones commented on RANGER-676: I've heard colleagues express an interest in Elastic Search too. Has there been any integration work on using Elastic search done to date? > Audit logging to Elasticsearch > -- > > Key: RANGER-676 > URL: https://issues.apache.org/jira/browse/RANGER-676 > Project: Ranger > Issue Type: New Feature > Components: audit >Affects Versions: 0.5.0 > Environment: HDP 2.3 >Reporter: Hari Sekhon > > Feature Request to have audit logging output to Elasticsearch since there is > an output to Solr but Elasticsearch is easier to operate at scale. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RANGER-1701) gaiandb: simple resource based policy
[ https://issues.apache.org/jira/browse/RANGER-1701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109072#comment-16109072 ] Nigel Jones commented on RANGER-1701: - this will include the resource mapper > gaiandb: simple resource based policy > - > > Key: RANGER-1701 > URL: https://issues.apache.org/jira/browse/RANGER-1701 > Project: Ranger > Issue Type: Sub-task > Components: plugins >Reporter: Nigel Jones > > Get the gaiandb plugin to a state where a simple resource based policy is > operational (permit/deny), and the ranger gui can be used to define.. and the > plugin will then implement this -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (RANGER-1720) gaiandb: resource lookup
Nigel Jones created RANGER-1720: --- Summary: gaiandb: resource lookup Key: RANGER-1720 URL: https://issues.apache.org/jira/browse/RANGER-1720 Project: Ranger Issue Type: Sub-task Components: plugins Reporter: Nigel Jones Add resource lookup for typeahead -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (RANGER-1719) Share ranger metadata with Atlas
Nigel Jones created RANGER-1719: --- Summary: Share ranger metadata with Atlas Key: RANGER-1719 URL: https://issues.apache.org/jira/browse/RANGER-1719 Project: Ranger Issue Type: New Feature Components: admin Reporter: Nigel Jones In ATLAS-1869 I propose that we capture various metadata from Ranger to make it available to atlas for stewardship, reporting, compliance. Opening this as a placeholder to discuss the nature of required plugin or bridge in ranger My thought is to keep this as loosely coupled as possible, so for example it could be a case of adding notifications so that ranger can publish changes in policies which a component related to atlas could pick up & mediate & inject into atlas. Ranger would remain the master for policy definitions, but once Atlas knows about them they could be linked with business terms in atlas & business-related policies, and we could drive governance reports using this data -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (RANGER-1454) Support of Atlas GAF OMAS
[ https://issues.apache.org/jira/browse/RANGER-1454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated RANGER-1454: Summary: Support of Atlas GAF OMAS (was: Support of Atlas v2 glossary API proposal for tag source) > Support of Atlas GAF OMAS > -- > > Key: RANGER-1454 > URL: https://issues.apache.org/jira/browse/RANGER-1454 > Project: Ranger > Issue Type: New Feature > Components: tagsync >Reporter: Nigel Jones >Assignee: Nigel Jones > Labels: VirtualDataConnector > > A new v2 glossary capability is proposed for Atlas in ATLAS-1410. > As part of this the glossary model becomes more sophisticated. In order to > preserve the current simple tag(trait type):parms(trait instance)-entity > relationship a new tagsync process will be developed that makes use of a new > API proposed in ATLAS-1662. This would be an alternative to the current one > so that existing users could continue unaffected, and a change is only > required if moving to the v2 glossary implementation > This will also allow the process of retrieving tags to be more efficient, and > this new API can form a consumer-centric interface to support multiple > enforcement technologies, including, but not restricted to, Ranger. > feel free to assign to me/modify permissions ;-) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RANGER-1488) Create Ranger plugin for gaiandb
[ https://issues.apache.org/jira/browse/RANGER-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16094711#comment-16094711 ] Nigel Jones commented on RANGER-1488: - I've created a high level breakdown of the stages I see in implementing this plugin. Currently the code on github hooks into gaian's policy framework and can intercept calls. There is also a simple configuration file defined (which I will post in the relevant task) but the code is not yet complete and does not initialize with ranger. this is the next step. Is the community ok with plugin code being contributed into the ranger codebase, as long as licensing is ok? If so I will aim to add proposed patches to the review board tool . It's probably reasonable to require the first 3 subtasks are completed before any code is submitted to the project since that is the first point of having some coherent that adds value, but I can add sooner if that's helpful? The work-in-progress can be viewed on github, but hopefully this is just a staging area There is some more information about the reason we're building this plugin which is currently being driven on the Atlas side. See https://cwiki.apache.org/confluence/display/ATLAS/GaianDB+as+a+virtualizer+with+governance+enforcement > Create Ranger plugin for gaiandb > > > Key: RANGER-1488 > URL: https://issues.apache.org/jira/browse/RANGER-1488 > Project: Ranger > Issue Type: New Feature > Components: plugins >Reporter: Nigel Jones >Assignee: Nigel Jones > Labels: VirtualDataConnector > > GaianDB is a distributed, federated database built on Apache Derby. > The documentation is at > https://github.com/gaiandb/gaiandb/blob/master/README.md - navigate up for > the source. > As part of a Virtual Connector Project based on Atlas we are using gaianDB to > provide a virtualization layer. We need to control access to underlying > resources and will be building a Ranger plugin for gaiandb to support this. > GaianDB already has support for a form of policy plugin which allows the SQL > to be intercepted. > It is unclear if this code will sit > - external to ranger/atlas ie in the gaianDB sources > - as a sample in the Atlas project (so that we collate what is needed for > the virtual connector project) > - as a sample, or component in the Ranger project > However it seems helpful to describe the proposal here and get > feedback/dialogue in the ranger community :-) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (RANGER-1699) gaiandb: Create initial plugin configuration file that can be deployed to ranger
[ https://issues.apache.org/jira/browse/RANGER-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated RANGER-1699: Summary: gaiandb: Create initial plugin configuration file that can be deployed to ranger (was: Create initial plugin configuration file that can be deployed to ranger) > gaiandb: Create initial plugin configuration file that can be deployed to > ranger > > > Key: RANGER-1699 > URL: https://issues.apache.org/jira/browse/RANGER-1699 > Project: Ranger > Issue Type: Sub-task > Components: plugins >Reporter: Nigel Jones > > Create initial configuration file for the gaiandb plugin for ranger which > will define policies, resources. > Additional jiras will be opened to add extra capabilities later on. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (RANGER-1703) gaiandb - add data masking (similar to hive) for resource based policy
Nigel Jones created RANGER-1703: --- Summary: gaiandb - add data masking (similar to hive) for resource based policy Key: RANGER-1703 URL: https://issues.apache.org/jira/browse/RANGER-1703 Project: Ranger Issue Type: Sub-task Components: plugins Reporter: Nigel Jones Support data masking in gaindb plugin. Initially implement for a resource based policy -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (RANGER-1704) gaiandb - add data masking support for tag based policy
Nigel Jones created RANGER-1704: --- Summary: gaiandb - add data masking support for tag based policy Key: RANGER-1704 URL: https://issues.apache.org/jira/browse/RANGER-1704 Project: Ranger Issue Type: Sub-task Components: plugins Reporter: Nigel Jones Support data masking of columns etc using a tag based policy -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (RANGER-1702) gaiandb: tag based policies
Nigel Jones created RANGER-1702: --- Summary: gaiandb: tag based policies Key: RANGER-1702 URL: https://issues.apache.org/jira/browse/RANGER-1702 Project: Ranger Issue Type: Sub-task Components: plugins Reporter: Nigel Jones Update plugin to allow tag based policies to be used for simple permit/deny -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (RANGER-1701) gaiandb: simple resource based policy
Nigel Jones created RANGER-1701: --- Summary: gaiandb: simple resource based policy Key: RANGER-1701 URL: https://issues.apache.org/jira/browse/RANGER-1701 Project: Ranger Issue Type: Sub-task Components: plugins Reporter: Nigel Jones Get the gaiandb plugin to a state where a simple resource based policy is operational (permit/deny), and the ranger gui can be used to define.. and the plugin will then implement this -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (RANGER-1700) gaiandb: Integrate with ranger libraries - plugin builds, initializes
Nigel Jones created RANGER-1700: --- Summary: gaiandb: Integrate with ranger libraries - plugin builds, initializes Key: RANGER-1700 URL: https://issues.apache.org/jira/browse/RANGER-1700 Project: Ranger Issue Type: Sub-task Components: plugins Reporter: Nigel Jones Get the plugin to a state where it will initialize with a simple configuration and can be deployed to gaianDB. It may not actually action any policies at this state -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (RANGER-1487) generate rules from Governance definitions in Atlas
[ https://issues.apache.org/jira/browse/RANGER-1487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16084003#comment-16084003 ] Nigel Jones edited comment on RANGER-1487 at 7/12/17 1:47 PM: -- Having considered this approach some more, ALTAS-1869 will now discuss a mechanism for adding the capability to capture some metadata about ranger rules within atlas. This can allow business concepts & organization policies to be linked with specific rules that implement these policies. I will add more info there and create an similar jira for ranger if we feel that info is useful in atlas was (Author: jonesn): Having considered this approach some more, RANGER-1869 will now discuss a mechanism for adding the capability to capture some metadata about ranger rules within atlas. This can allow business concepts & organization policies to be linked with specific rules that implement these policies > generate rules from Governance definitions in Atlas > --- > > Key: RANGER-1487 > URL: https://issues.apache.org/jira/browse/RANGER-1487 > Project: Ranger > Issue Type: New Feature > Components: Ranger >Reporter: Nigel Jones >Assignee: Nigel Jones > Labels: VirtualDataConnector > > As part of the Atlas virtual connector project we intend to build "rules" > that relate to classifications within Atlas itself. These are logical > definitions that will then be mapped to specific implementations such as > rules/policies in ranger (but could also be in other systems like Oracle, or > Apache Sentry -- different enforcement approaches for different data access > points). > Opening this Jira to build a "sync/build/link" tool that will generate ranger > rules where possible. > (Very rough description ... will update & clarify) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RANGER-1487) generate rules from Governance definitions in Atlas
[ https://issues.apache.org/jira/browse/RANGER-1487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16084003#comment-16084003 ] Nigel Jones commented on RANGER-1487: - Having considered this approach some more, RANGER-1869 will now discuss a mechanism for adding the capability to capture some metadata about ranger rules within atlas. This can allow business concepts & organization policies to be linked with specific rules that implement these policies > generate rules from Governance definitions in Atlas > --- > > Key: RANGER-1487 > URL: https://issues.apache.org/jira/browse/RANGER-1487 > Project: Ranger > Issue Type: New Feature > Components: Ranger >Reporter: Nigel Jones >Assignee: Nigel Jones > Labels: VirtualDataConnector > > As part of the Atlas virtual connector project we intend to build "rules" > that relate to classifications within Atlas itself. These are logical > definitions that will then be mapped to specific implementations such as > rules/policies in ranger (but could also be in other systems like Oracle, or > Apache Sentry -- different enforcement approaches for different data access > points). > Opening this Jira to build a "sync/build/link" tool that will generate ranger > rules where possible. > (Very rough description ... will update & clarify) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (RANGER-1487) generate rules from Governance definitions in Atlas
[ https://issues.apache.org/jira/browse/RANGER-1487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones resolved RANGER-1487. - Resolution: Fixed > generate rules from Governance definitions in Atlas > --- > > Key: RANGER-1487 > URL: https://issues.apache.org/jira/browse/RANGER-1487 > Project: Ranger > Issue Type: New Feature > Components: Ranger >Reporter: Nigel Jones >Assignee: Nigel Jones > Labels: VirtualDataConnector > > As part of the Atlas virtual connector project we intend to build "rules" > that relate to classifications within Atlas itself. These are logical > definitions that will then be mapped to specific implementations such as > rules/policies in ranger (but could also be in other systems like Oracle, or > Apache Sentry -- different enforcement approaches for different data access > points). > Opening this Jira to build a "sync/build/link" tool that will generate ranger > rules where possible. > (Very rough description ... will update & clarify) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RANGER-1454) Support of Atlas v2 glossary API proposal for tag source
[ https://issues.apache.org/jira/browse/RANGER-1454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16036957#comment-16036957 ] Nigel Jones commented on RANGER-1454: - correct, thanks for spotting. the ATLAS-1662 I referred to does refer to the governance action OMAS, but the abstract of this Jira is badly named. It is to support the new glossary implementation in Atlas, but that is exposed to ranger through the governanance action OMAS. I'll update the title. > Support of Atlas v2 glossary API proposal for tag source > > > Key: RANGER-1454 > URL: https://issues.apache.org/jira/browse/RANGER-1454 > Project: Ranger > Issue Type: New Feature > Components: tagsync >Reporter: Nigel Jones >Assignee: Nigel Jones > Labels: VirtualDataConnector > > A new v2 glossary capability is proposed for Atlas in ATLAS-1410. > As part of this the glossary model becomes more sophisticated. In order to > preserve the current simple tag(trait type):parms(trait instance)-entity > relationship a new tagsync process will be developed that makes use of a new > API proposed in ATLAS-1662. This would be an alternative to the current one > so that existing users could continue unaffected, and a change is only > required if moving to the v2 glossary implementation > This will also allow the process of retrieving tags to be more efficient, and > this new API can form a consumer-centric interface to support multiple > enforcement technologies, including, but not restricted to, Ranger. > feel free to assign to me/modify permissions ;-) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (RANGER-1572) Provide simple example of running ranger with docker
[ https://issues.apache.org/jira/browse/RANGER-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012031#comment-16012031 ] Nigel Jones commented on RANGER-1572: - the latest example on github will now correctly setup and start ranger to the point of being able to login to the admin UI Continuing > Provide simple example of running ranger with docker > > > Key: RANGER-1572 > URL: https://issues.apache.org/jira/browse/RANGER-1572 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Nigel Jones >Assignee: Nigel Jones > > When experimenting with Ranger it's very useful to be able to quickly run > ranger admin/solr/mysql > Since I needed this for my education/experimentation I thought it might be > useful to others. Work in progress and doesn't work yet but raising this JIRA > in case anyone else has done the same, or is interested ;-) > For now the code (just a few scripts) is at > https://github.com/planetf1/ranger-docker whilst I try and get this working. > Perhaps this could be added as an example down the line, or even used within > the build to create an image -- though there's lots of variables in terms of > config. > As above.. not working yet ;-) WIP! -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (RANGER-1186) Ranger Source: eclipse
[ https://issues.apache.org/jira/browse/RANGER-1186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005069#comment-16005069 ] Nigel Jones commented on RANGER-1186: - As an aside... I may look back at this, but for now am using Intelli-J as it's... simpler.. less fighting the IDE ;-) > Ranger Source: eclipse > -- > > Key: RANGER-1186 > URL: https://issues.apache.org/jira/browse/RANGER-1186 > Project: Ranger > Issue Type: Bug > Components: documentation >Affects Versions: 0.7.0 > Environment: eclipse neon, ubuntu 16.10 >Reporter: Nigel Jones >Assignee: Pradeep Agrawal > Labels: VirtualDataConnector > Attachments: RANGER-1186_Ranger_Source_Eclipse.pdf, ranger_me0.PNG, > ranger_me1.PNG > > > I can happily build ranger in my Ubuntu 16.10 environment with > - open jdk 1.8.0_102 > - maven 3.3.9 > I also have eclipse neon installed in this environment > The build/setup instructions at > http://ranger.apache.org/quick_start_guide.html have minimal instructions for > eclipse. > I imported the top level ranger directory using the m2e plugin as provided in > neon. > This failed with numerous errors > Is anyone actively building with eclipse? If so can we update the website > with current setup information? > If not I'll use this JIRA to record the issues & hope to address them & > update the docs ;-) > Screenshots to follow -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (RANGER-1454) Support of Atlas v2 glossary API proposal for tag source
[ https://issues.apache.org/jira/browse/RANGER-1454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005066#comment-16005066 ] Nigel Jones commented on RANGER-1454: - Yes I've raised this as a placeholder for now. Once the glossary itself becomes clearer, then I believe we should have a new API on Atlas which exposes the tags in a ranger-friendly form (also the subject of a related jira). The v2 glossary is more sophisticated so there are multiple levels of classifications .. but the new API (pointed to by a new tagsync, or a variant thereof) can allow ranger to work the same as today. The discussion after that may be to consider whether any additional metadata should be available in ranger as a first-class entity so that it can easily be used in rules... but we're a fair few steps away from that ;-) > Support of Atlas v2 glossary API proposal for tag source > > > Key: RANGER-1454 > URL: https://issues.apache.org/jira/browse/RANGER-1454 > Project: Ranger > Issue Type: New Feature > Components: tagsync >Reporter: Nigel Jones >Assignee: Nigel Jones > Labels: VirtualDataConnector > > A new v2 glossary capability is proposed for Atlas in ATLAS-1410. > As part of this the glossary model becomes more sophisticated. In order to > preserve the current simple tag(trait type):parms(trait instance)-entity > relationship a new tagsync process will be developed that makes use of a new > API proposed in ATLAS-1662. This would be an alternative to the current one > so that existing users could continue unaffected, and a change is only > required if moving to the v2 glossary implementation > This will also allow the process of retrieving tags to be more efficient, and > this new API can form a consumer-centric interface to support multiple > enforcement technologies, including, but not restricted to, Ranger. > feel free to assign to me/modify permissions ;-) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (RANGER-1487) generate rules from Governance definitions in Atlas
[ https://issues.apache.org/jira/browse/RANGER-1487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones reassigned RANGER-1487: --- Assignee: Nigel Jones > generate rules from Governance definitions in Atlas > --- > > Key: RANGER-1487 > URL: https://issues.apache.org/jira/browse/RANGER-1487 > Project: Ranger > Issue Type: New Feature > Components: Ranger >Reporter: Nigel Jones >Assignee: Nigel Jones > Labels: VirtualDataConnector > > As part of the Atlas virtual connector project we intend to build "rules" > that relate to classifications within Atlas itself. These are logical > definitions that will then be mapped to specific implementations such as > rules/policies in ranger (but could also be in other systems like Oracle, or > Apache Sentry -- different enforcement approaches for different data access > points). > Opening this Jira to build a "sync/build/link" tool that will generate ranger > rules where possible. > (Very rough description ... will update & clarify) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (RANGER-1563) Ranger build needs gcc... docs/validation
[ https://issues.apache.org/jira/browse/RANGER-1563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones reassigned RANGER-1563: --- Assignee: Nigel Jones > Ranger build needs gcc... docs/validation > - > > Key: RANGER-1563 > URL: https://issues.apache.org/jira/browse/RANGER-1563 > Project: Ranger > Issue Type: Bug > Components: documentation >Reporter: Nigel Jones >Assignee: Nigel Jones > > When building ranger on a new Linux system I had a build failure, caused by > [INFO] Executed tasks > [INFO] > [INFO] --- native-maven-plugin:1.0-alpha-8:compile (default-compile) @ > credValidator --- > [INFO] /bin/sh -c cd /home/cloudusr/src/ranger/unixauthnative && gcc > -I/home/cloudusr/src/ranger/unixauthnative/src/main/c -o > /home/cloudusr/src/ranger/unixauthnative/target/objs/credValidator.o -c > /home/cloudusr/src/ranger/unixauthnative/src/main/c/credValidator.c > /bin/sh: 1: gcc: not found > Clearly gcc is needed. Along with java, maven. Listing these prereqs would be > useful > Suggest we update the docs at http://ranger.apache.org/quick_start_guide.html > to make this prereq clear. > Ideally update maven to do a check early on -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (RANGER-1565) Make SOLR setup config docs consistent with code
[ https://issues.apache.org/jira/browse/RANGER-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones reassigned RANGER-1565: --- Assignee: Nigel Jones > Make SOLR setup config docs consistent with code > > > Key: RANGER-1565 > URL: https://issues.apache.org/jira/browse/RANGER-1565 > Project: Ranger > Issue Type: Bug > Components: audit >Reporter: Nigel Jones >Assignee: Nigel Jones > > In the Apache ranger setup docs we have a section on Solr configuration at > https://cwiki.apache.org/confluence/display/RANGER/Install+and+Configure+Solr+for+Ranger+Audits+-+Apache+Ranger+0.5 > In this doc it refers to setting JAVA_HOME in the install.properties file > (the one in the contrib/ solr setup directory). However when I set this the > ./setup.sh still failed with: > # ./setup.sh > Error: JAVA_HOME environment property not defined, aborting installation. > Looking at the code it appears it does NOT pull out JAVA_HOME from the > properties file at all, but relies on it being specified in the environment > Ideally we would update the script to also source JAVA_HOME from > install.properties, in which case the doc can stay as is. If we don't do this > we need to update the docs accordingly. I would go with the properties file > overriding the local environment, if the variable is set there. > My proposal is to change the script. Happy to work this if it sounds ok... -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (RANGER-1572) Provide simple example of running ranger with docker
[ https://issues.apache.org/jira/browse/RANGER-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones reassigned RANGER-1572: --- Assignee: Nigel Jones > Provide simple example of running ranger with docker > > > Key: RANGER-1572 > URL: https://issues.apache.org/jira/browse/RANGER-1572 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Nigel Jones >Assignee: Nigel Jones > > When experimenting with Ranger it's very useful to be able to quickly run > ranger admin/solr/mysql > Since I needed this for my education/experimentation I thought it might be > useful to others. Work in progress and doesn't work yet but raising this JIRA > in case anyone else has done the same, or is interested ;-) > For now the code (just a few scripts) is at > https://github.com/planetf1/ranger-docker whilst I try and get this working. > Perhaps this could be added as an example down the line, or even used within > the build to create an image -- though there's lots of variables in terms of > config. > As above.. not working yet ;-) WIP! -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (RANGER-1572) Provide simple example of running ranger with docker
Nigel Jones created RANGER-1572: --- Summary: Provide simple example of running ranger with docker Key: RANGER-1572 URL: https://issues.apache.org/jira/browse/RANGER-1572 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Nigel Jones When experimenting with Ranger it's very useful to be able to quickly run ranger admin/solr/mysql Since I needed this for my education/experimentation I thought it might be useful to others. Work in progress and doesn't work yet but raising this JIRA in case anyone else has done the same, or is interested ;-) For now the code (just a few scripts) is at https://github.com/planetf1/ranger-docker whilst I try and get this working. Perhaps this could be added as an example down the line, or even used within the build to create an image -- though there's lots of variables in terms of config. As above.. not working yet ;-) WIP! -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (RANGER-1565) Make SOLR setup config docs consistent with code
[ https://issues.apache.org/jira/browse/RANGER-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated RANGER-1565: Request participants: (was: ) Description: In the Apache ranger setup docs we have a section on Solr configuration at https://cwiki.apache.org/confluence/display/RANGER/Install+and+Configure+Solr+for+Ranger+Audits+-+Apache+Ranger+0.5 In this doc it refers to setting JAVA_HOME in the install.properties file (the one in the contrib/ solr setup directory). However when I set this the ./setup.sh still failed with: # ./setup.sh Error: JAVA_HOME environment property not defined, aborting installation. Looking at the code it appears it does NOT pull out JAVA_HOME from the properties file at all, but relies on it being specified in the environment Ideally we would update the script to also source JAVA_HOME from install.properties, in which case the doc can stay as is. If we don't do this we need to update the docs accordingly. I would go with the properties file overriding the local environment, if the variable is set there. My proposal is to change the script. Happy to work this if it sounds ok... was: In the Apache ranger setup docs we have a section on Solr configuration at https://cwiki.apache.org/confluence/display/RANGER/Install+and+Configure+Solr+for+Ranger+Audits+-+Apache+Ranger+0.5 In this doc it refers to setting JAVA_HOME in the install.properties file (the one in the contrib/ solr setup directory). However when I set this the ./setup.sh still failed with: # ./setup.sh Error: JAVA_HOME environment property not defined, aborting installation. Looking at the code it appears it does NOT pull out JAVA_HOME from the properties file at all, but relies on it being specified in the environment Ideally we would update the script to also source JAVA_HOME from install.properties, in which case the doc can stay as is. If we don't do this we need to update the docs accordingly. My proposal is to change the script. Happy to work this if it sounds ok... > Make SOLR setup config docs consistent with code > > > Key: RANGER-1565 > URL: https://issues.apache.org/jira/browse/RANGER-1565 > Project: Ranger > Issue Type: Bug > Components: audit >Reporter: Nigel Jones > > In the Apache ranger setup docs we have a section on Solr configuration at > https://cwiki.apache.org/confluence/display/RANGER/Install+and+Configure+Solr+for+Ranger+Audits+-+Apache+Ranger+0.5 > In this doc it refers to setting JAVA_HOME in the install.properties file > (the one in the contrib/ solr setup directory). However when I set this the > ./setup.sh still failed with: > # ./setup.sh > Error: JAVA_HOME environment property not defined, aborting installation. > Looking at the code it appears it does NOT pull out JAVA_HOME from the > properties file at all, but relies on it being specified in the environment > Ideally we would update the script to also source JAVA_HOME from > install.properties, in which case the doc can stay as is. If we don't do this > we need to update the docs accordingly. I would go with the properties file > overriding the local environment, if the variable is set there. > My proposal is to change the script. Happy to work this if it sounds ok... -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (RANGER-1565) Make SOLR setup config docs consistent with code
Nigel Jones created RANGER-1565: --- Summary: Make SOLR setup config docs consistent with code Key: RANGER-1565 URL: https://issues.apache.org/jira/browse/RANGER-1565 Project: Ranger Issue Type: Bug Components: audit Reporter: Nigel Jones In the Apache ranger setup docs we have a section on Solr configuration at https://cwiki.apache.org/confluence/display/RANGER/Install+and+Configure+Solr+for+Ranger+Audits+-+Apache+Ranger+0.5 In this doc it refers to setting JAVA_HOME in the install.properties file (the one in the contrib/ solr setup directory). However when I set this the ./setup.sh still failed with: # ./setup.sh Error: JAVA_HOME environment property not defined, aborting installation. Looking at the code it appears it does NOT pull out JAVA_HOME from the properties file at all, but relies on it being specified in the environment Ideally we would update the script to also source JAVA_HOME from install.properties, in which case the doc can stay as is. If we don't do this we need to update the docs accordingly. My proposal is to change the script. Happy to work this if it sounds ok... -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (RANGER-1563) Ranger build needs gcc... docs/validation
[ https://issues.apache.org/jira/browse/RANGER-1563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000594#comment-16000594 ] Nigel Jones commented on RANGER-1563: - Ok I'll ask again. But happy to propose this change, good practice for next time :-) > Ranger build needs gcc... docs/validation > - > > Key: RANGER-1563 > URL: https://issues.apache.org/jira/browse/RANGER-1563 > Project: Ranger > Issue Type: Bug > Components: documentation >Reporter: Nigel Jones > > When building ranger on a new Linux system I had a build failure, caused by > [INFO] Executed tasks > [INFO] > [INFO] --- native-maven-plugin:1.0-alpha-8:compile (default-compile) @ > credValidator --- > [INFO] /bin/sh -c cd /home/cloudusr/src/ranger/unixauthnative && gcc > -I/home/cloudusr/src/ranger/unixauthnative/src/main/c -o > /home/cloudusr/src/ranger/unixauthnative/target/objs/credValidator.o -c > /home/cloudusr/src/ranger/unixauthnative/src/main/c/credValidator.c > /bin/sh: 1: gcc: not found > Clearly gcc is needed. Along with java, maven. Listing these prereqs would be > useful > Suggest we update the docs at http://ranger.apache.org/quick_start_guide.html > to make this prereq clear. > Ideally update maven to do a check early on -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (RANGER-1563) Ranger build needs gcc... docs/validation
[ https://issues.apache.org/jira/browse/RANGER-1563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000584#comment-16000584 ] Nigel Jones edited comment on RANGER-1563 at 5/8/17 10:46 AM: -- Of course, I'd be delighted to (though first time may take a while). related -- I did post on the mailing list asking for JIRA Contributor, so that I can assign to myself - are you able to do this? If not I'll post again was (Author: jonesn): Of course, I'd be delighted too (though first time may take a while). related -- I did post on the mailing list asking for JIRA Contributor, so that I can assign to myself - are you able to do this? If not I'll post again > Ranger build needs gcc... docs/validation > - > > Key: RANGER-1563 > URL: https://issues.apache.org/jira/browse/RANGER-1563 > Project: Ranger > Issue Type: Bug > Components: documentation >Reporter: Nigel Jones > > When building ranger on a new Linux system I had a build failure, caused by > [INFO] Executed tasks > [INFO] > [INFO] --- native-maven-plugin:1.0-alpha-8:compile (default-compile) @ > credValidator --- > [INFO] /bin/sh -c cd /home/cloudusr/src/ranger/unixauthnative && gcc > -I/home/cloudusr/src/ranger/unixauthnative/src/main/c -o > /home/cloudusr/src/ranger/unixauthnative/target/objs/credValidator.o -c > /home/cloudusr/src/ranger/unixauthnative/src/main/c/credValidator.c > /bin/sh: 1: gcc: not found > Clearly gcc is needed. Along with java, maven. Listing these prereqs would be > useful > Suggest we update the docs at http://ranger.apache.org/quick_start_guide.html > to make this prereq clear. > Ideally update maven to do a check early on -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (RANGER-1563) Ranger build needs gcc... docs/validation
[ https://issues.apache.org/jira/browse/RANGER-1563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000584#comment-16000584 ] Nigel Jones commented on RANGER-1563: - Of course, I'd be delighted too (though first time may take a while). related -- I did post on the mailing list asking for JIRA Contributor, so that I can assign to myself - are you able to do this? If not I'll post again > Ranger build needs gcc... docs/validation > - > > Key: RANGER-1563 > URL: https://issues.apache.org/jira/browse/RANGER-1563 > Project: Ranger > Issue Type: Bug > Components: documentation >Reporter: Nigel Jones > > When building ranger on a new Linux system I had a build failure, caused by > [INFO] Executed tasks > [INFO] > [INFO] --- native-maven-plugin:1.0-alpha-8:compile (default-compile) @ > credValidator --- > [INFO] /bin/sh -c cd /home/cloudusr/src/ranger/unixauthnative && gcc > -I/home/cloudusr/src/ranger/unixauthnative/src/main/c -o > /home/cloudusr/src/ranger/unixauthnative/target/objs/credValidator.o -c > /home/cloudusr/src/ranger/unixauthnative/src/main/c/credValidator.c > /bin/sh: 1: gcc: not found > Clearly gcc is needed. Along with java, maven. Listing these prereqs would be > useful > Suggest we update the docs at http://ranger.apache.org/quick_start_guide.html > to make this prereq clear. > Ideally update maven to do a check early on -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (RANGER-1563) Ranger build needs gcc... docs/validation
Nigel Jones created RANGER-1563: --- Summary: Ranger build needs gcc... docs/validation Key: RANGER-1563 URL: https://issues.apache.org/jira/browse/RANGER-1563 Project: Ranger Issue Type: Bug Components: documentation Reporter: Nigel Jones When building ranger on a new Linux system I had a build failure, caused by [INFO] Executed tasks [INFO] [INFO] --- native-maven-plugin:1.0-alpha-8:compile (default-compile) @ credValidator --- [INFO] /bin/sh -c cd /home/cloudusr/src/ranger/unixauthnative && gcc -I/home/cloudusr/src/ranger/unixauthnative/src/main/c -o /home/cloudusr/src/ranger/unixauthnative/target/objs/credValidator.o -c /home/cloudusr/src/ranger/unixauthnative/src/main/c/credValidator.c /bin/sh: 1: gcc: not found Clearly gcc is needed. Along with java, maven. Listing these prereqs would be useful Suggest we update the docs at http://ranger.apache.org/quick_start_guide.html to make this prereq clear. Ideally update maven to do a check early on -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (RANGER-1488) Create Ranger plugin for gaiandb
[ https://issues.apache.org/jira/browse/RANGER-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000517#comment-16000517 ] Nigel Jones commented on RANGER-1488: - Initial code being put together at https://github.com/planetf1/gaiandb-policy-ranger ... it will be a few weeks before this is in a functional state, but I wanted to give a heads up about the activity > Create Ranger plugin for gaiandb > > > Key: RANGER-1488 > URL: https://issues.apache.org/jira/browse/RANGER-1488 > Project: Ranger > Issue Type: New Feature > Components: plugins >Reporter: Nigel Jones > Labels: VirtualDataConnector > > GaianDB is a distributed, federated database built on Apache Derby. > The documentation is at > https://github.com/gaiandb/gaiandb/blob/master/README.md - navigate up for > the source. > As part of a Virtual Connector Project based on Atlas we are using gaianDB to > provide a virtualization layer. We need to control access to underlying > resources and will be building a Ranger plugin for gaiandb to support this. > GaianDB already has support for a form of policy plugin which allows the SQL > to be intercepted. > It is unclear if this code will sit > - external to ranger/atlas ie in the gaianDB sources > - as a sample in the Atlas project (so that we collate what is needed for > the virtual connector project) > - as a sample, or component in the Ranger project > However it seems helpful to describe the proposal here and get > feedback/dialogue in the ranger community :-) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (RANGER-1186) Ranger Source: eclipse
[ https://issues.apache.org/jira/browse/RANGER-1186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated RANGER-1186: Request participants: (was: ) Labels: VirtualDataConnector (was: ) > Ranger Source: eclipse > -- > > Key: RANGER-1186 > URL: https://issues.apache.org/jira/browse/RANGER-1186 > Project: Ranger > Issue Type: Bug > Components: documentation >Affects Versions: 0.7.0 > Environment: eclipse neon, ubuntu 16.10 >Reporter: Nigel Jones >Assignee: Pradeep Agrawal > Labels: VirtualDataConnector > Attachments: RANGER-1186_Ranger_Source_Eclipse.pdf, ranger_me0.PNG, > ranger_me1.PNG > > > I can happily build ranger in my Ubuntu 16.10 environment with > - open jdk 1.8.0_102 > - maven 3.3.9 > I also have eclipse neon installed in this environment > The build/setup instructions at > http://ranger.apache.org/quick_start_guide.html have minimal instructions for > eclipse. > I imported the top level ranger directory using the m2e plugin as provided in > neon. > This failed with numerous errors > Is anyone actively building with eclipse? If so can we update the website > with current setup information? > If not I'll use this JIRA to record the issues & hope to address them & > update the docs ;-) > Screenshots to follow -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (RANGER-1168) Add data masking for tag based policies
[ https://issues.apache.org/jira/browse/RANGER-1168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15949226#comment-15949226 ] Nigel Jones commented on RANGER-1168: - I will be building a plugin for gaianDB and need to evaluate if any changes are needed in ranger to support for a new plugin. > Add data masking for tag based policies > --- > > Key: RANGER-1168 > URL: https://issues.apache.org/jira/browse/RANGER-1168 > Project: Ranger > Issue Type: New Feature >Reporter: Nigel Jones > > Currently in Hive its possible to create tag based policies. For example a > policy can prevent a user from accessing data in a column tagged as sensitive > - with that classification likely coming from Apache Atlas. > In order to support the principle of applying a policy to a classification > rather than an actual resort, we should extend Ranger & in particular the > plugins such as hive to allow tag based policies to support > - data masking > - row filtering > I believe this is important to support the principle of Apache Atlas being > used to centrally manage metadata & the classification of resources -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (RANGER-1488) Create Ranger plugin for gaiandb
Nigel Jones created RANGER-1488: --- Summary: Create Ranger plugin for gaiandb Key: RANGER-1488 URL: https://issues.apache.org/jira/browse/RANGER-1488 Project: Ranger Issue Type: New Feature Components: plugins Reporter: Nigel Jones GaianDB is a distributed, federated database built on Apache Derby. The documentation is at https://github.com/gaiandb/gaiandb/blob/master/README.md - navigate up for the source. As part of a Virtual Connector Project based on Atlas we are using gaianDB to provide a virtualization layer. We need to control access to underlying resources and will be building a Ranger plugin for gaiandb to support this. GaianDB already has support for a form of policy plugin which allows the SQL to be intercepted. It is unclear if this code will sit - external to ranger/atlas ie in the gaianDB sources - as a sample in the Atlas project (so that we collate what is needed for the virtual connector project) - as a sample, or component in the Ranger project However it seems helpful to describe the proposal here and get feedback/dialogue in the ranger community :-) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (RANGER-1464) Support Atlas v2 glossary in Atlas plugin (for access control to terms etc)
[ https://issues.apache.org/jira/browse/RANGER-1464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated RANGER-1464: Request participants: (was: ) Labels: VirtualDataConnector (was: ) > Support Atlas v2 glossary in Atlas plugin (for access control to terms etc) > --- > > Key: RANGER-1464 > URL: https://issues.apache.org/jira/browse/RANGER-1464 > Project: Ranger > Issue Type: New Feature > Components: plugins >Reporter: Nigel Jones > Labels: VirtualDataConnector > > A v2 glossary is proposed in ATLAS-1410 > The atlas ranger plugin should be updated to support access control of the > new glossary support once/if this atlas change is completed similarly (or > indeed improved over) to the v1 term/taxonomy support. > Will add more details as v2 glossary evolves. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (RANGER-1485) Ranger support for Virtual Data Connector Project (ATLAS)
[ https://issues.apache.org/jira/browse/RANGER-1485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated RANGER-1485: Request participants: (was: ) Labels: VirtualDataConnector (was: ) > Ranger support for Virtual Data Connector Project (ATLAS) > - > > Key: RANGER-1485 > URL: https://issues.apache.org/jira/browse/RANGER-1485 > Project: Ranger > Issue Type: New Feature > Components: plugins >Reporter: Nigel Jones > Labels: VirtualDataConnector > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (RANGER-1487) generate rules from Governance definitions in Atlas
[ https://issues.apache.org/jira/browse/RANGER-1487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated RANGER-1487: Request participants: (was: ) Labels: VirtualDataConnector (was: ) > generate rules from Governance definitions in Atlas > --- > > Key: RANGER-1487 > URL: https://issues.apache.org/jira/browse/RANGER-1487 > Project: Ranger > Issue Type: New Feature > Components: Ranger >Reporter: Nigel Jones > Labels: VirtualDataConnector > > As part of the Atlas virtual connector project we intend to build "rules" > that relate to classifications within Atlas itself. These are logical > definitions that will then be mapped to specific implementations such as > rules/policies in ranger (but could also be in other systems like Oracle, or > Apache Sentry -- different enforcement approaches for different data access > points). > Opening this Jira to build a "sync/build/link" tool that will generate ranger > rules where possible. > (Very rough description ... will update & clarify) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (RANGER-1487) generate rules from Governance definitions in Atlas
Nigel Jones created RANGER-1487: --- Summary: generate rules from Governance definitions in Atlas Key: RANGER-1487 URL: https://issues.apache.org/jira/browse/RANGER-1487 Project: Ranger Issue Type: New Feature Components: Ranger Reporter: Nigel Jones As part of the Atlas virtual connector project we intend to build "rules" that relate to classifications within Atlas itself. These are logical definitions that will then be mapped to specific implementations such as rules/policies in ranger (but could also be in other systems like Oracle, or Apache Sentry -- different enforcement approaches for different data access points). Opening this Jira to build a "sync/build/link" tool that will generate ranger rules where possible. (Very rough description ... will update & clarify) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (RANGER-1486) New usersync alternative for Atlas (vdc)
Nigel Jones created RANGER-1486: --- Summary: New usersync alternative for Atlas (vdc) Key: RANGER-1486 URL: https://issues.apache.org/jira/browse/RANGER-1486 Project: Ranger Issue Type: New Feature Components: usersync Reporter: Nigel Jones As part of the Atlas Virtualization Data Connector work we are using this within a large enterprise with a lot of users & groups stored in ldap. The connector -- which has a ranger plugin to apply access control policies -- is used by a relatively small subset of these users. However that can't easily be transcribed to an optimal ldap query. Since Atlas will have the definitive list of roles that are being used, this new usersync will instead retrieve a list of roles from Atlas, and will then use this list to retrieve only those users found in this list of roles from LDAP. This is an alternative usersync so shouldn't conflict and will use the same ranger APIs -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (RANGER-1485) Ranger support for Virtual Data Connector Project (ATLAS)
Nigel Jones created RANGER-1485: --- Summary: Ranger support for Virtual Data Connector Project (ATLAS) Key: RANGER-1485 URL: https://issues.apache.org/jira/browse/RANGER-1485 Project: Ranger Issue Type: New Feature Components: plugins Reporter: Nigel Jones -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (RANGER-1464) Support Atlas v2 glossary in Atlas plugin (for access control to terms etc)
Nigel Jones created RANGER-1464: --- Summary: Support Atlas v2 glossary in Atlas plugin (for access control to terms etc) Key: RANGER-1464 URL: https://issues.apache.org/jira/browse/RANGER-1464 Project: Ranger Issue Type: New Feature Components: plugins Reporter: Nigel Jones A v2 glossary is proposed in ATLAS-1410 The atlas ranger plugin should be updated to support access control of the new glossary support once/if this atlas change is completed similarly (or indeed improved over) to the v1 term/taxonomy support. Will add more details as v2 glossary evolves. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (RANGER-1331) Ranger build for windows not working
[ https://issues.apache.org/jira/browse/RANGER-1331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851480#comment-15851480 ] Nigel Jones commented on RANGER-1331: - Made an error with not having python installed. Having been installed ( Python 2.7.13 ) mvn -Dmaven.test.skip=true clean install completes successfully As does mvn -Dmaven.test.skip=true clean compile package install assembly:assembly It is only the tests (!) that fail. The issues seem to include pathname/parsing issues and lack of unix shell Can we use this jira to update the web instructions so that it refers to skipping unit tests in this way on windows - perhaps recommends running those on linux - and that it also reminds the user to install python, as well as git/maven/java - the latter are more obvious but sometimes for newbies to the project it's easier to be clear. I'm happy to propose some updates/patch if it helps Are there any other users around that use windows and feel it would be useful to fix up the tests? It may be not? > Ranger build for windows not working > > > Key: RANGER-1331 > URL: https://issues.apache.org/jira/browse/RANGER-1331 > Project: Ranger > Issue Type: Improvement >Reporter: Nigel Jones > Attachments: RangerWinBuild1.log > > > The quick start guide at http://ranger.apache.org/quick_start_guide.html has > instuctions to build ranger which work fine on Ubuntu 16.04/openjdk 8 / maven > 3.3.9 > However I was interested in setting up a more productive IDE environment, and > given my laptop is running Windows 10 as the primary OS decided to follow the > instructions for using eclipse > HOWEVER unlike Ranger-1186 I tried this on *WINDOWS* rather than linux > Is this something anyone is doing successfully (even if skipping building > some of the linux native auth pieces)? > I then tried the build but it failed quickly on credential support > [INFO] ranger . SUCCESS [ 9.332 > s] > [INFO] Jdbc SQL Connector . SUCCESS [ 4.569 > s] > [INFO] Credential Support . FAILURE [ 9.579 > s] > Including full log > Could we list on that page the OSs people have had success with? > Are most using linux? > Anyone tried MacOS? > NOTE: edited from original to focus on the OS question rather than IDE -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (RANGER-1186) Ranger Source: eclipse
[ https://issues.apache.org/jira/browse/RANGER-1186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15850158#comment-15850158 ] Nigel Jones commented on RANGER-1186: - Remembering there are still missing directories - next issue to look at. Happy to have this assigned and propose some doc changes in a patch->review board [it will be be a good first step for me] if that helps ? > Ranger Source: eclipse > -- > > Key: RANGER-1186 > URL: https://issues.apache.org/jira/browse/RANGER-1186 > Project: Ranger > Issue Type: Bug > Components: documentation >Affects Versions: Ranger > Environment: eclipse neon, ubuntu 16.10 >Reporter: Nigel Jones >Assignee: Pradeep Agrawal > Attachments: RANGER-1186_Ranger_Source_Eclipse.pdf, ranger_me0.PNG, > ranger_me1.PNG > > > I can happily build ranger in my Ubuntu 16.10 environment with > - open jdk 1.8.0_102 > - maven 3.3.9 > I also have eclipse neon installed in this environment > The build/setup instructions at > http://ranger.apache.org/quick_start_guide.html have minimal instructions for > eclipse. > I imported the top level ranger directory using the m2e plugin as provided in > neon. > This failed with numerous errors > Is anyone actively building with eclipse? If so can we update the website > with current setup information? > If not I'll use this JIRA to record the issues & hope to address them & > update the docs ;-) > Screenshots to follow -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (RANGER-1186) Ranger Source: eclipse
[ https://issues.apache.org/jira/browse/RANGER-1186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15850109#comment-15850109 ] Nigel Jones commented on RANGER-1186: - A quick update on my attempt to get eclipse running more cleanly. Via https://github.com/mojohaus/build-helper-maven-plugin/issues/43 I can see that after testing prerelease, simply updating pom.xml with a new version for the build helper plugin, along with taking the eclipse quick-fix to pull in lifecycle mappers (a normal technique) will fix the errors relating to that plugin org.codehaus.mojo build-helper-maven-plugin 3.0.0 Then we're left with plugins that are yet to become m2e compatible - see referenced defects for that discussion. I opted to force their tasks to run during build. This felt the most appropriate rather than ignore - since this might not find issues introduced whilst coding. For this I added some entries to plugin management. It may not be perfect, but it's a starting point, resulting in what seems like a clean (no error) eclipse environment org.eclipse.m2e lifecycle-mapping 1.0.0 org.apache.maven.plugins maven-antrun-plugin [1.7,) run org.codehaus.mojo native-maven-plugin [1.0-alpha-8,) compile org.codehaus.mojo native-maven-plugin [1.0-alpha-8,) initialize link
[jira] [Updated] (RANGER-1331) Ranger build for windows not working
[ https://issues.apache.org/jira/browse/RANGER-1331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated RANGER-1331: Attachment: RangerWinBuild1.log > Ranger build for windows not working > > > Key: RANGER-1331 > URL: https://issues.apache.org/jira/browse/RANGER-1331 > Project: Ranger > Issue Type: Improvement >Reporter: Nigel Jones > Attachments: RangerWinBuild1.log > > > The quick start guide at http://ranger.apache.org/quick_start_guide.html has > instuctions to build ranger which work fine on Ubuntu 16.04/openjdk 8 / maven > 3.3.9 > However I was interested in setting up a more productive IDE environment, and > given my laptop is running Windows 10 as the primary OS decided to follow the > instructions for using eclipse > HOWEVER unlike Ranger-1186 I tried this on *WINDOWS* rather than linux > Is this something anyone is doing successfully (even if skipping building > some of the linux native auth pieces)? > I then tried the build but it failed quickly on credential support > [INFO] ranger . SUCCESS [ 9.332 > s] > [INFO] Jdbc SQL Connector . SUCCESS [ 4.569 > s] > [INFO] Credential Support . FAILURE [ 9.579 > s] > Including full log > Could we list on that page the OSs people have had success with? > Are most using linux? > Anyone tried MacOS? > NOTE: edited from original to focus on the OS question rather than IDE -- This message was sent by Atlassian JIRA (v6.3.4#6332)