[jira] [Commented] (RANGER-1480) Implement plugin for Druid
[ https://issues.apache.org/jira/browse/RANGER-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314807#comment-17314807 ] Bolke de Bruin commented on RANGER-1480: You are aware that Druid ships with a Ranger plugin for some time now right? > Implement plugin for Druid > -- > > Key: RANGER-1480 > URL: https://issues.apache.org/jira/browse/RANGER-1480 > Project: Ranger > Issue Type: New Feature > Components: admin, plugins >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Attachments: > 0001-RANGER-1480-Second-version-of-the-druid-plugin.patch, Screen Shot > 2021-01-19 at 16.36.06.png > > > Druid is a high-performance, column-oriented, distributed data store, which > has an extension mechanism to incorporate various functionalities, for > example an external authorization system - just like ranger. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-1480) Implement plugin for Druid
[ https://issues.apache.org/jira/browse/RANGER-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17268142#comment-17268142 ] Bolke de Bruin commented on RANGER-1480: The official druid release has a ranger plugin nowadays > Implement plugin for Druid > -- > > Key: RANGER-1480 > URL: https://issues.apache.org/jira/browse/RANGER-1480 > Project: Ranger > Issue Type: New Feature > Components: admin, plugins >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Attachments: 0001-RANGER-1480-Second-version-of-the-druid-plugin.patch > > > Druid is a high-performance, column-oriented, distributed data store, which > has an extension mechanism to incorporate various functionalities, for > example an external authorization system - just like ranger. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (RANGER-2850) Add javax.annotation to Presto plugin to ensure java 11 support
[ https://issues.apache.org/jira/browse/RANGER-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated RANGER-2850: --- External issue URL: https://reviews.apache.org/r/72572/ > Add javax.annotation to Presto plugin to ensure java 11 support > --- > > Key: RANGER-2850 > URL: https://issues.apache.org/jira/browse/RANGER-2850 > Project: Ranger > Issue Type: Improvement > Components: plugins > Reporter: Bolke de Bruin > Assignee: Bolke de Bruin >Priority: Major > Attachments: > 0001-RANGER-2850-Ensure-inclusion-of-javax.annotations-in.patch > > > Presto's assembly is missing the annotation jar which makes it not work with > Java 11. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (RANGER-2754) Update presto dependency and implement row/column level security
[ https://issues.apache.org/jira/browse/RANGER-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17127284#comment-17127284 ] Bolke de Bruin edited comment on RANGER-2754 at 6/6/20, 8:25 AM: - [~ppanda-beta] we run presto 332 with java 8. We havent tested java 11 yet and that might indeed require patches. Latest master was updated to support the additional controls that presto 332 required, which indeed checks for function and procedure execution rights. You will need to update your service definitions and policies to ensure it works. was (Author: bolke): [~ppanda-beta] we run presto 332 with java 8. We havent tested java 11 yet and that might indeed require patches. > Update presto dependency and implement row/column level security > > > Key: RANGER-2754 > URL: https://issues.apache.org/jira/browse/RANGER-2754 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: master > Reporter: Bolke de Bruin >Assignee: Bolke de Bruin >Priority: Major > Fix For: 2.1.0 > > Attachments: > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > RANGER-2754-v2.patch, RANGER-2754.patch > > > 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped > working for versions > ~321. > 2. Presto master now has row/column level security support -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2754) Update presto dependency and implement row/column level security
[ https://issues.apache.org/jira/browse/RANGER-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17127284#comment-17127284 ] Bolke de Bruin commented on RANGER-2754: [~ppanda-beta] we run presto 332 with java 8. We havent tested java 11 yet and that might indeed require patches. > Update presto dependency and implement row/column level security > > > Key: RANGER-2754 > URL: https://issues.apache.org/jira/browse/RANGER-2754 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: master > Reporter: Bolke de Bruin >Assignee: Bolke de Bruin >Priority: Major > Fix For: 2.1.0 > > Attachments: > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > RANGER-2754-v2.patch, RANGER-2754.patch > > > 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped > working for versions > ~321. > 2. Presto master now has row/column level security support -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2844) presto-server-333 can not show schemas/tables after integrated with Ranger master branch
[
https://issues.apache.org/jira/browse/RANGER-2844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17127274#comment-17127274
]
Bolke de Bruin commented on RANGER-2844:
Please ensure you have applied the latest servicedefinition. ‘Show’ is in
master and if you do not have it in the UI you probably have an outdated
service definitiojn
> presto-server-333 can not show schemas/tables after integrated with Ranger
> master branch
>
>
> Key: RANGER-2844
> URL: https://issues.apache.org/jira/browse/RANGER-2844
> Project: Ranger
> Issue Type: Bug
> Components: Ranger
>Affects Versions: 2.1.0
> Environment: Amazon Linux
>Reporter: Jie Zhang
>Assignee: Pradeep Agrawal
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: got access denied for show command.png, no show
> command.png
>
>
> Integrated presto-server-333 with Ranger master branch, able to run
> use/select queries, but got access denied when running *{color:#00875a}show
> schemas/tables{color}*.
> I saw there is no {color:#00875a}*show*{color} command on ranger-ui
> (screenshot attached), is that why presto can not show schemas/tables? Any
> workaround? Thanks a lot.
>
> Error:
> {code:java}
> presto:default> use default;
> USE
> presto:default> show schemas;
> Query 20200604_192311_00023_sa7kt failed: Access Denied: Cannot show schemas:
> hivepresto:default> show tables;
> Query 20200604_192319_00024_sa7kt failed: Access Denied: Cannot show tables
> of schema hive.default
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (RANGER-1784) Feature Request: Presto Ranger integration
[ https://issues.apache.org/jira/browse/RANGER-1784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated RANGER-1784: --- Fix Version/s: 2.1 > Feature Request: Presto Ranger integration > -- > > Key: RANGER-1784 > URL: https://issues.apache.org/jira/browse/RANGER-1784 > Project: Ranger > Issue Type: New Feature > Components: admin, audit, plugins, Ranger >Affects Versions: 0.7.1 > Environment: HDP 2.6 + Kerberos + Presto >Reporter: Hari Sekhon >Priority: Critical > Fix For: 2.1 > > > Feature Request to add Presto authorization support to Ranger. > This could perhaps piggy back to use Hive policies in Ranger or else copy > what Hive has to a separately managed policy for Presto. There are trade-offs > to both styles, so perhaps make this a configurable user choice. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (RANGER-2192) Implement Presto plugin
[ https://issues.apache.org/jira/browse/RANGER-2192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin resolved RANGER-2192. Fix Version/s: 2.1.0 Resolution: Fixed > Implement Presto plugin > --- > > Key: RANGER-2192 > URL: https://issues.apache.org/jira/browse/RANGER-2192 > Project: Ranger > Issue Type: New Feature > Components: plugins, Ranger >Reporter: t oo >Priority: Major > Fix For: 2.1.0 > > > Implement Presto plugin -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (RANGER-2816) Presto-ranger plugin has jersey error on presto server startup
[
https://issues.apache.org/jira/browse/RANGER-2816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bolke de Bruin resolved RANGER-2816.
Resolution: Won't Fix
Support for Presto was added in Ranger 2.
> Presto-ranger plugin has jersey error on presto server startup
> --
>
> Key: RANGER-2816
> URL: https://issues.apache.org/jira/browse/RANGER-2816
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Affects Versions: 1.0.0
> Environment: amazonlinux1, java8 (OpenJDK Runtime Environment (build
> 1.8.0_242-b08)). Looking at what has changed in the environment:
> yum packages -->
> from
> file.x86_64 5.37-8.48.amzn1 installed
>
> file-libs.x86_64 5.37-8.48.amzn1 installed
> kernel.x86_64 4.14.165-102.185.amzn1 installed
>
> kernel.x86_64 4.14.171-105.231.amzn1
> @amzn-updates
> kernel-headers.x86_64 4.14.171-105.231.amzn1
> @amzn-updates
> libicu.x86_64 50.1.2-11.12.amzn1 installed
>
> libtirpc.x86_64 0.2.4-0.8.14.amzn1 installed
>
>
> to
> file.x86_64 5.37-8.49.amzn1 installed
> file-libs.x86_64 5.37-8.49.amzn1 installed
> kernel.x86_64 4.14.171-105.231.amzn1 installed
> kernel.x86_64 4.14.173-106.229.amzn1
> @amzn-updates
> kernel-headers.x86_64 4.14.173-106.229.amzn1
> @amzn-updates
> libicu.x86_64 50.2-4.0.amzn1
> @amzn-updates
> libtirpc.x86_64 0.2.4-0.16.15.amzn1
> @amzn-updates
> jersey jars:
> /home/ec2-user/presto-server-0.220/lib/jersey-media-jaxb-2.26.jar
> /home/ec2-user/presto-server-0.220/lib/jersey-container-servlet-2.26.jar
> /home/ec2-user/presto-server-0.220/lib/jersey-hk2-2.26.jar
> /home/ec2-user/presto-server-0.220/lib/jersey-server-2.26.jar
> /home/ec2-user/presto-server-0.220/lib/jersey-client-2.26.jar
> /home/ec2-user/presto-server-0.220/lib/jersey-container-servlet-core-2.26.jar
> /home/ec2-user/presto-server-0.220/lib/jersey-common-2.26.jar
> /home/ec2-user/presto-server-0.220/plugin/ranger/ranger-presto-plugin-impl/jersey-server-1.9.jar
> /home/ec2-user/presto-server-0.220/plugin/ranger/ranger-presto-plugin-impl/jersey-bundle-1.19.3.jar
> /home/ec2-user/presto-server-0.220/plugin/ranger/ranger-presto-plugin-impl/jersey-json-1.9.jar
> /home/ec2-user/presto-server-0.220/plugin/ranger/ranger-presto-plugin-impl/jersey-core-1.9.jar
> /usr/lib/apache-ranger-1.0.0/target/ranger-1.0.0-usersync/lib/jersey-bundle-1.19.3.jar
> /usr/lib/apache-ranger-1.0.0/target/ranger-1.0.0-presto-plugin/lib/ranger-presto-plugin-impl/jersey-server-1.9.jar
> /usr/lib/apache-ranger-1.0.0/target/ranger-1.0.0-presto-plugin/lib/ranger-presto-plugin-impl/jersey-bundle-1.19.3.jar
> /usr/lib/apache-ranger-1.0.0/target/ranger-1.0.0-presto-plugin/lib/ranger-presto-plugin-impl/jersey-json-1.9.jar
> /usr/lib/apache-ranger-1.0.0/target/ranger-1.0.0-presto-plugin/lib/ranger-presto-plugin-impl/jersey-core-1.9.jar
> /usr/lib/apache-ranger-1.0.0/target/ranger-1.0.0-admin/ews/webapp/WEB-INF/lib/jersey-server-1.9.jar
> /usr/lib/apache-ranger-1.0.0/target/ranger-1.0.0-admin/ews/webapp/WEB-INF/lib/jersey-bundle-1.19.3.jar
> /usr/lib/apache-ranger-1.0.0/target/ranger-1.0.0-admin/ews/webapp/WEB-INF/lib/jersey-core-1.19.3.jar
> /usr/lib/apache-ranger-1.0.0/target/ranger-1.0.0-admin/ews/webapp/WEB-INF/lib/jersey-client-1.9.jar
> /usr/lib/apache-ranger-1.0.0/target/ranger-1.0.0-admin/ews/webapp/WEB-INF/lib/jersey-multipart-1.19.3.jar
> /usr/lib/apache-ranger-1.0.0/target/ranger-1.0.0-admin/ews/webapp/WEB-INF/lib/jersey-spring-1.19.3.jar
> /usr/lib/apache-ranger-1.0.0/target/ranger-1.0.0-admin/ews/webapp/WEB-INF/lib/jersey-json-1.9.jar
> /usr/lib/apache-ranger-1.0.0/target/ranger-1.0.0-admin/ews/webapp/WEB-INF/lib/jersey-guice-1.9.jar
> /usr/lib/apache-ranger-1.0.0/target/ranger-1.0.0-admin/ews/webapp/WEB-INF/lib/jersey-servlet-1.19.3.jar
>Reporter: t oo
>Priority: Major
> Attachments: patch2.diff
>
>
> I am using ranger 1.0.0 and prestodb 220, everything has been working for >
> 10 mths until today presto startup consistently gets this error:
> {code:java}
> 2020-05-04T18:50:26.036ZINFOmain
> org.apache.ranger.audit.queue.AuditFileSpoolStarting writerThread,
> queueName
[jira] [Resolved] (RANGER-1784) Feature Request: Presto Ranger integration
[ https://issues.apache.org/jira/browse/RANGER-1784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin resolved RANGER-1784. Resolution: Fixed Ranger has support for PrestoSQL > Feature Request: Presto Ranger integration > -- > > Key: RANGER-1784 > URL: https://issues.apache.org/jira/browse/RANGER-1784 > Project: Ranger > Issue Type: New Feature > Components: admin, audit, plugins, Ranger >Affects Versions: 0.7.1 > Environment: HDP 2.6 + Kerberos + Presto >Reporter: Hari Sekhon >Priority: Critical > > Feature Request to add Presto authorization support to Ranger. > This could perhaps piggy back to use Hive policies in Ranger or else copy > what Hive has to a separately managed policy for Presto. There are trade-offs > to both styles, so perhaps make this a configurable user choice. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (RANGER-2850) Add javax.annotation to Presto plugin to ensure java 11 support
Bolke de Bruin created RANGER-2850: -- Summary: Add javax.annotation to Presto plugin to ensure java 11 support Key: RANGER-2850 URL: https://issues.apache.org/jira/browse/RANGER-2850 Project: Ranger Issue Type: Improvement Components: plugins Reporter: Bolke de Bruin Assignee: Bolke de Bruin Presto's assembly is missing the annotation jar which makes it not work with Java 11. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2835) Presto plugin DROP TABLE or DELETE is always denied
[ https://issues.apache.org/jira/browse/RANGER-2835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17127350#comment-17127350 ] Bolke de Bruin commented on RANGER-2835: [~toopt4] I'm not seeing an issue here? You are getting an access denied on "drop table" which could be in your policy? Remember the default is to deny. Did you upgrade your service definition which is required when going to > 332. > Presto plugin DROP TABLE or DELETE is always denied > --- > > Key: RANGER-2835 > URL: https://issues.apache.org/jira/browse/RANGER-2835 > Project: Ranger > Issue Type: Bug > Components: plugins, Ranger >Reporter: t oo >Priority: Major > > DROP VIEW works, INSERT works but DROP TABLE does not work, DELETE does not > work > > > "com.facebook.presto.spi.security.AccessDeniedException.denyDropTable(AccessDeniedException.java:111)", > > "com.facebook.presto.spi.security.AccessDeniedException.denyDropTable(AccessDeniedException.java:106)", > > "com.facebook.presto.hive.security.LegacyAccessControl.checkCanDropTable(LegacyAccessControl.java:99)", > > "com.facebook.presto.hive.security.PartitionsAwareAccessControl.checkCanDropTable(PartitionsAwareAccessControl.java:88)", > > "com.facebook.presto.security.AccessControlManager.lambda$checkCanDropTable$19(AccessControlManager.java:283)", > > "com.facebook.presto.security.AccessControlManager.authorizationCheck(AccessControlManager.java:717)", > > "com.facebook.presto.security.AccessControlManager.checkCanDropTable(AccessControlManager.java:283)", > > "com.facebook.presto.execution.DropTableTask.execute(DropTableTask.java:57)", > "com.facebook.presto.execution.DropTableTask.execute(DropTableTask.java:34)", > "com.facebook.presto.execution.DataDefinitionExecution.start(DataDefinitionExecution.java:169)", > > "java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)", > > "java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)", > "java.lang.Thread.run(Thread.java:748)" > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (RANGER-2850) Add javax.annotation to Presto plugin to ensure java 11 support
[ https://issues.apache.org/jira/browse/RANGER-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated RANGER-2850: --- Attachment: 0001-RANGER-2850-Ensure-inclusion-of-javax.annotations-in.patch > Add javax.annotation to Presto plugin to ensure java 11 support > --- > > Key: RANGER-2850 > URL: https://issues.apache.org/jira/browse/RANGER-2850 > Project: Ranger > Issue Type: Improvement > Components: plugins > Reporter: Bolke de Bruin > Assignee: Bolke de Bruin >Priority: Major > Attachments: > 0001-RANGER-2850-Ensure-inclusion-of-javax.annotations-in.patch > > > Presto's assembly is missing the annotation jar which makes it not work with > Java 11. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (RANGER-2609) delete permission not present on presto service def json
[ https://issues.apache.org/jira/browse/RANGER-2609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin resolved RANGER-2609. Resolution: Fixed > delete permission not present on presto service def json > > > Key: RANGER-2609 > URL: https://issues.apache.org/jira/browse/RANGER-2609 > Project: Ranger > Issue Type: Bug > Components: plugins >Affects Versions: 2.0.0, 2.1.0 >Reporter: Pedro Rossi >Priority: Major > > in here > ([https://github.com/apache/ranger/blob/master/plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java#L274)] > the code check if the user can delete from table and that is ok, but here > ([https://github.com/apache/ranger/blob/master/agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json#L97)] > the code doesn't expose the delete access to the ranger ui which leads all > delete from table queries to be denied -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (RANGER-2747) Ranger integration with presto works on "Release 317" but fails on versions later
[ https://issues.apache.org/jira/browse/RANGER-2747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin resolved RANGER-2747. Resolution: Fixed fixed in master > Ranger integration with presto works on "Release 317" but fails on versions > later > - > > Key: RANGER-2747 > URL: https://issues.apache.org/jira/browse/RANGER-2747 > Project: Ranger > Issue Type: Bug > Components: plugins >Reporter: harsha >Priority: Blocker > Time Spent: 20m > Remaining Estimate: 0h > > Hi, > I am trying to integrate ranger with presto and its working on "Release 317" > I followed the following documentation : > [https://cwiki.apache.org/confluence/display/RANGER/Presto+Plugin] > But, when when I go with versions after that, It throws the following errors. > *query*: _SHOW CATALOGS_ > *error* : _[65536] Query failed (#20200303_063413_2_h5s89): Internal > error java.lang.RuntimeException: java.lang.IllegalStateException_ > *query*:_select * from hive.db_name.table_name limit 10_ > *error*:_[4] Query failed (#20200303_063836_0_x47vg): Access Denied: > Cannot access catalog hive java.lang.RuntimeException: > io.prestosql.spi.security.AccessDeniedException: Access Denied: Cannot access > catalog hive_ > I've tried versions 318,319,323,329 and i am facing similar errors. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (RANGER-2483) Presto plugin uses /presto/plugins instead of /presto/plugin
[ https://issues.apache.org/jira/browse/RANGER-2483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin resolved RANGER-2483. Resolution: Fixed Fixed in master > Presto plugin uses /presto/plugins instead of /presto/plugin > > > Key: RANGER-2483 > URL: https://issues.apache.org/jira/browse/RANGER-2483 > Project: Ranger > Issue Type: Bug > Components: plugins >Affects Versions: 2.0.0 >Reporter: David Berger >Priority: Major > > Presto plugin uses /presto/plugins instead of /presto/plugin > The workaround is to create a symlink before enabling the plugin. > ln -s /presto/plugin presto/plugins > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (RANGER-2472) Presto-plugin error upon setup
[
https://issues.apache.org/jira/browse/RANGER-2472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bolke de Bruin resolved RANGER-2472.
Resolution: Duplicate
> Presto-plugin error upon setup
> --
>
> Key: RANGER-2472
> URL: https://issues.apache.org/jira/browse/RANGER-2472
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Affects Versions: 2.0.0
>Reporter: Pedro Rossi
>Priority: Major
>
> * The {{ranger-presto-*.xml}} files must be packed into a jar and added to
> {{/plugins/ranger/ranger-presto-impl}} in order to be recognized by the
> {{...getClassLoader().getResource("...")}} line, is there any other way to
> add these files without having to pack a jar?
> * After the {{*.xml}} files are added it gives an error about missing this
> lib ([https://github.com/mattsheppard/gethostname4j]) upon using reflection
> but after adding it to the same folder above it worked fine, shouldn't this
> lib be on added automatically upon building the project?
> Notes: I used the docker script to build the project and later moved the jars
> to my presto build docker container
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (RANGER-2716) Add catalogs/schemas/tables filter in presto plugin
[ https://issues.apache.org/jira/browse/RANGER-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin resolved RANGER-2716. Resolution: Duplicate > Add catalogs/schemas/tables filter in presto plugin > --- > > Key: RANGER-2716 > URL: https://issues.apache.org/jira/browse/RANGER-2716 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: Jiayi Liu >Priority: Major > Attachments: 0001-RANGER-2716.patch, 0002-RANGER-2716.patch > > > Presto plugin returns the input set of catalogs/schemas/tables directly at > present. However, this causes a problem, that is, when the user uses show > catalogs/schemas/tables, all catalogs/schemas/tables are displayed, even > though the user does not have the permissions to display all > catalogs/schemas/tables. > We need to fix the filterCatalogs/Schemas/Tables functions to filter > catalogs/schemas/tables, so that the user can only see the > catalogs/schemas/tables in which the user has SELECT permission. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Review Request 72572: Ensure javax.annotations is present in presto assembly to support java 11
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72572/ --- Review request for ranger, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, and Ramesh Mani. Bugs: RANGER-2850 https://issues.apache.org/jira/browse/RANGER-2850 Repository: ranger Description --- javax.annotations was not correctly added to the presto assembly and thus missing Diffs - distro/src/main/assembly/plugin-presto.xml 508fbfa71 Diff: https://reviews.apache.org/r/72572/diff/1/ Testing --- - build & deploy Thanks, Bolke de Bruin
[jira] [Commented] (RANGER-2754) Update presto dependency and implement row/column level security
[ https://issues.apache.org/jira/browse/RANGER-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110094#comment-17110094 ] Bolke de Bruin commented on RANGER-2754: It's already fixed in master > Update presto dependency and implement row/column level security > > > Key: RANGER-2754 > URL: https://issues.apache.org/jira/browse/RANGER-2754 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: master >Reporter: Bolke de Bruin >Assignee: Bolke de Bruin >Priority: Major > Fix For: 2.1.0 > > Attachments: > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > RANGER-2754-v2.patch, RANGER-2754.patch > > > 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped > working for versions > ~321. > 2. Presto master now has row/column level security support -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 72513: Add support for Presto 333
> On May 15, 2020, 2:43 p.m., Madhan Neethiraj wrote: > > agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json > > Lines 164 (patched) > > <https://reviews.apache.org/r/72513/diff/1/?file=2231715#file2231715line164> > > > > Changes in service-def (like addition of resources/access-types, > > changes in access-types/data-mask/row-filter) will not be applied to > > existing service-def in Ranger. Updating existing service-def would require > > a Java patch. Please file a JIRA to track Java patch to handle this. RANGER-2795 is tracking this > On May 15, 2020, 2:43 p.m., Madhan Neethiraj wrote: > > agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json > > Lines 266 (patched) > > <https://reviews.apache.org/r/72513/diff/1/?file=2231715#file2231715line266> > > > > I suggest to not change itemId of existing entries; and assign > > itemId=13 for the new entry 'execute'. Sure. Is itemid used anywhere? > On May 15, 2020, 2:43 p.m., Madhan Neethiraj wrote: > > plugin-presto/src/test/resources/presto-policies.json > > Line 1013 (original), 1187 (patched) > > <https://reviews.apache.org/r/72513/diff/1/?file=2231718#file2231718line1187> > > > > Row-filter expressions are likely to refer to columns in the table for > > which the filter is applied. When wildcard is allowed in row-filter > > policies, it might be challenging to make sure that the row-filter > > expression is valid for all the tables covered by the wildcard. When the > > row-filter is invalid for a table, the query will fail. Hence wildCard was > > disabled for row-filters. Please review to make sure that this is clearly > > understood. I understand your point, however, this is already in the master, you are commenting on an update to the test policies which are a reflection of what is currently in master. - Bolke --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72513/#review220783 --- On May 17, 2020, 7:38 a.m., Bolke de Bruin wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/72513/ > --- > > (Updated May 17, 2020, 7:38 a.m.) > > > Review request for ranger, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, > and Ramesh Mani. > > > Bugs: https://jira.apache.org/jira/browse/RANGER-2826 > > https://issues.apache.org/jira/browse/https://jira.apache.org/jira/browse/RANGER-2826 > > > Repository: ranger > > > Description > --- > > Presto 332/333 are backwards incompatible. > > > Diffs > - > > agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json > 4d5b79582 > > plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java > d4521a392 > > plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControlTest.java > c00d51986 > plugin-presto/src/test/resources/presto-policies.json 28eabf2d6 > pom.xml ebce7c9f0 > > ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java > bfb3a5961 > > security-admin/src/main/java/org/apache/ranger/patch/PatchForPrestoToSupportPresto333_J10037.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/72513/diff/2/ > > > Testing > --- > > Unit tests updated. Production. > > > Thanks, > > Bolke de Bruin > >
Re: Review Request 72513: Add support for Presto 333
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72513/ --- (Updated May 17, 2020, 7:38 a.m.) Review request for ranger, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, and Ramesh Mani. Changes --- - Added upgrade patch - Addressed comments Bugs: https://jira.apache.org/jira/browse/RANGER-2826 https://issues.apache.org/jira/browse/https://jira.apache.org/jira/browse/RANGER-2826 Repository: ranger Description --- Presto 332/333 are backwards incompatible. Diffs (updated) - agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json 4d5b79582 plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java d4521a392 plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControlTest.java c00d51986 plugin-presto/src/test/resources/presto-policies.json 28eabf2d6 pom.xml ebce7c9f0 ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java bfb3a5961 security-admin/src/main/java/org/apache/ranger/patch/PatchForPrestoToSupportPresto333_J10037.java PRE-CREATION Diff: https://reviews.apache.org/r/72513/diff/2/ Changes: https://reviews.apache.org/r/72513/diff/1-2/ Testing --- Unit tests updated. Production. Thanks, Bolke de Bruin
Re: Review Request 72513: Add support for Presto 333
> On May 15, 2020, 3:57 a.m., Pradeep Agrawal wrote: > > Presto plugin was added in ranger-2.0. Will these changes be available for > > users who upgrade from ranger-2.0 ? What will be the behaviour with this > > change in ranger upgrade case. the plugin itself is backwards compatible with older Prestos. note I recently committed an uodate to this plugin "after ranger 2.0" tbis is merely a follow up. - Bolke --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72513/#review220777 --- On May 14, 2020, 8:25 p.m., Bolke de Bruin wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/72513/ > --- > > (Updated May 14, 2020, 8:25 p.m.) > > > Review request for ranger, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, > and Ramesh Mani. > > > Bugs: https://jira.apache.org/jira/browse/RANGER-2826 > > https://issues.apache.org/jira/browse/https://jira.apache.org/jira/browse/RANGER-2826 > > > Repository: ranger > > > Description > --- > > Presto 332/333 are backwards incompatible. > > > Diffs > - > > agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json > 4d5b79582 > > plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java > d4521a392 > > plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControlTest.java > c00d51986 > plugin-presto/src/test/resources/presto-policies.json 28eabf2d6 > pom.xml ebce7c9f0 > > ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java > bfb3a5961 > > > Diff: https://reviews.apache.org/r/72513/diff/1/ > > > Testing > --- > > Unit tests updated. Production. > > > Thanks, > > Bolke de Bruin > >
[jira] [Updated] (RANGER-2826) Update Presto Plugin to support PrestoSql 333
[ https://issues.apache.org/jira/browse/RANGER-2826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated RANGER-2826: --- External issue URL: https://reviews.apache.org/r/72513/ > Update Presto Plugin to support PrestoSql 333 > - > > Key: RANGER-2826 > URL: https://issues.apache.org/jira/browse/RANGER-2826 > Project: Ranger > Issue Type: Improvement > Components: plugins > Reporter: Bolke de Bruin >Priority: Major > Attachments: 0001-RANGER-2826-Add-support-for-PrestoSQL-333.patch > > > PrestoSql has updated its security API again and made it backwards > incompatible. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2826) Update Presto Plugin to support PrestoSql 333
[ https://issues.apache.org/jira/browse/RANGER-2826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17107649#comment-17107649 ] Bolke de Bruin commented on RANGER-2826: cc [~mad...@apache.org] > Update Presto Plugin to support PrestoSql 333 > - > > Key: RANGER-2826 > URL: https://issues.apache.org/jira/browse/RANGER-2826 > Project: Ranger > Issue Type: Improvement > Components: plugins > Reporter: Bolke de Bruin >Priority: Major > Attachments: 0001-RANGER-2826-Add-support-for-PrestoSQL-333.patch > > > PrestoSql has updated its security API again and made it backwards > incompatible. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Review Request 72513: Add support for Presto 333
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72513/ --- Review request for ranger, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, and Ramesh Mani. Bugs: https://jira.apache.org/jira/browse/RANGER-2826 https://issues.apache.org/jira/browse/https://jira.apache.org/jira/browse/RANGER-2826 Repository: ranger Description --- Presto 332/333 are backwards incompatible. Diffs - agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json 4d5b79582 plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java d4521a392 plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControlTest.java c00d51986 plugin-presto/src/test/resources/presto-policies.json 28eabf2d6 pom.xml ebce7c9f0 ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java bfb3a5961 Diff: https://reviews.apache.org/r/72513/diff/1/ Testing --- Unit tests updated. Production. Thanks, Bolke de Bruin
[jira] [Updated] (RANGER-2826) Update Presto Plugin to support PrestoSql 333
[ https://issues.apache.org/jira/browse/RANGER-2826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated RANGER-2826: --- Attachment: 0001-RANGER-2826-Add-support-for-PrestoSQL-333.patch > Update Presto Plugin to support PrestoSql 333 > - > > Key: RANGER-2826 > URL: https://issues.apache.org/jira/browse/RANGER-2826 > Project: Ranger > Issue Type: Improvement > Components: plugins > Reporter: Bolke de Bruin >Priority: Major > Attachments: 0001-RANGER-2826-Add-support-for-PrestoSQL-333.patch > > > PrestoSql has updated its security API again and made it backwards > incompatible. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (RANGER-2826) Update Presto Plugin to support PrestoSql 333
[ https://issues.apache.org/jira/browse/RANGER-2826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated RANGER-2826: --- Summary: Update Presto Plugin to support PrestoSql 333 (was: Update Presto Plugin to support PresoSql 333) > Update Presto Plugin to support PrestoSql 333 > - > > Key: RANGER-2826 > URL: https://issues.apache.org/jira/browse/RANGER-2826 > Project: Ranger > Issue Type: Improvement > Components: plugins > Reporter: Bolke de Bruin >Priority: Major > > PrestoSql has updated its security API again and made it backwards > incompatible. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (RANGER-2826) Update Presto Plugin to support PresoSql 333
Bolke de Bruin created RANGER-2826: -- Summary: Update Presto Plugin to support PresoSql 333 Key: RANGER-2826 URL: https://issues.apache.org/jira/browse/RANGER-2826 Project: Ranger Issue Type: Improvement Components: plugins Reporter: Bolke de Bruin PrestoSql has updated its security API again and made it backwards incompatible. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2716) Add catalogs/schemas/tables filter in presto plugin
[ https://issues.apache.org/jira/browse/RANGER-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17090539#comment-17090539 ] Bolke de Bruin commented on RANGER-2716: Awesome. I do wonder what those missing features are as I tried to be pretty comprehensive > Add catalogs/schemas/tables filter in presto plugin > --- > > Key: RANGER-2716 > URL: https://issues.apache.org/jira/browse/RANGER-2716 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: Jiayi Liu >Priority: Major > Attachments: 0001-RANGER-2716.patch, 0002-RANGER-2716.patch > > > Presto plugin returns the input set of catalogs/schemas/tables directly at > present. However, this causes a problem, that is, when the user uses show > catalogs/schemas/tables, all catalogs/schemas/tables are displayed, even > though the user does not have the permissions to display all > catalogs/schemas/tables. > We need to fix the filterCatalogs/Schemas/Tables functions to filter > catalogs/schemas/tables, so that the user can only see the > catalogs/schemas/tables in which the user has SELECT permission. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2716) Add catalogs/schemas/tables filter in presto plugin
[ https://issues.apache.org/jira/browse/RANGER-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17090473#comment-17090473 ] Bolke de Bruin commented on RANGER-2716: I suggest closing this as RANGER-2754 is now merged and addresses this. > Add catalogs/schemas/tables filter in presto plugin > --- > > Key: RANGER-2716 > URL: https://issues.apache.org/jira/browse/RANGER-2716 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: Jiayi Liu >Priority: Major > Attachments: 0001-RANGER-2716.patch, 0002-RANGER-2716.patch > > > Presto plugin returns the input set of catalogs/schemas/tables directly at > present. However, this causes a problem, that is, when the user uses show > catalogs/schemas/tables, all catalogs/schemas/tables are displayed, even > though the user does not have the permissions to display all > catalogs/schemas/tables. > We need to fix the filterCatalogs/Schemas/Tables functions to filter > catalogs/schemas/tables, so that the user can only see the > catalogs/schemas/tables in which the user has SELECT permission. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2754) Update presto dependency and implement row/column level security
[ https://issues.apache.org/jira/browse/RANGER-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17090474#comment-17090474 ] Bolke de Bruin commented on RANGER-2754: patch has been merged thanks [~madhan] > Update presto dependency and implement row/column level security > > > Key: RANGER-2754 > URL: https://issues.apache.org/jira/browse/RANGER-2754 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: master > Reporter: Bolke de Bruin >Assignee: Bolke de Bruin >Priority: Major > Attachments: > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > RANGER-2754-v2.patch, RANGER-2754.patch > > > 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped > working for versions > ~321. > 2. Presto master now has row/column level security support -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 72272: Upgrade and improve Presto plugin
> On apr 22, 2020, 3:55 p.m., Madhan Neethiraj wrote: > > Ship It! Can it be merged now then? - Bolke --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72272/#review220428 --- On apr 21, 2020, 11:43 a.m., Bolke de Bruin wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/72272/ > --- > > (Updated apr 21, 2020, 11:43 a.m.) > > > Review request for ranger, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, > and Ramesh Mani. > > > Bugs: https://issues.apache.org/jira/browse/RANGER-2754 > > https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/RANGER-2754 > > > Repository: ranger > > > Description > --- > > Upgrade and improve Presto plugin > - Presto SQL 331 has changed its security API and has Row level / column > masking functionality > - Upgraded Hadoop dependency to 3.1.3 (from 3.1.1) due to improved security > handling > - New features like session properties and system properties > > > Diffs > - > > agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json > 56a8f5ac0 > distro/src/main/assembly/plugin-presto.xml d2075bfe7 > plugin-presto/pom.xml b63f7dede > > plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java > 3ab63f590 > > plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerAdminClientImpl.java > PRE-CREATION > > plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControlTest.java > PRE-CREATION > plugin-presto/src/test/resources/log4j.properties PRE-CREATION > plugin-presto/src/test/resources/presto-policies.json PRE-CREATION > plugin-presto/src/test/resources/ranger-presto-security.xml PRE-CREATION > pom.xml 992c3654e > ranger-presto-plugin-shim/pom.xml d8ff88d0f > > ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerConfig.java > 67b0d2434 > > ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java > e89f646e1 > > > Diff: https://reviews.apache.org/r/72272/diff/10/ > > > Testing > --- > > - New Unit tests added > - Tested locally in production > > > Thanks, > > Bolke de Bruin > >
Fwd: Re: Review Request 72272: Upgrade and improve Presto plugin
Hi Madhan, Would you mind reviewing again? If you know how to ‘fix’ the assembly to ensure everything is included without putting it in the pom I will gladly do that. Cheers Bolke On 21 April 2020 at 13:45:36, Bolke de Bruin (bdbr...@gmail.com) wrote: This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72272/ On April 5th, 2020, 6:09 p.m. UTC, *Madhan Neethiraj* wrote: agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json <https://reviews.apache.org/r/72272/diff/2/?file=2215810#file2215810line96> (Diff revision 2) 96 }, To update existing Ranger instances with these service-def changes, consider adding a Java patch similar to PatchForAtlasToAddEntityLabelAndBusinessMetadata_J10034.java that handles recent updates to Atlas service-def. This can be taken up in a subsequent commit. On April 15th, 2020, 3:04 p.m. UTC, *Pradeep Agrawal* wrote: Created RANGER-2795 to track this separately. Thx. Will pick it up after merging this. On April 5th, 2020, 6:09 p.m. UTC, *Madhan Neethiraj* wrote: ranger-presto-plugin-shim/pom.xml <https://reviews.apache.org/r/72272/diff/2/?file=2215820#file2215820line93> (Diff revision 2) 93 org.apache.hadoop The changes in this module don't seem to refer hadoop-hdfs library contents directly. Is it necessary to explicitly add this dependency? On April 10th, 2020, 8:53 a.m. UTC, *Bolke de Bruin* wrote: I'm not getting it into the assembly otherwise for auditing purposes? please advise. On April 15th, 2020, 3:04 p.m. UTC, *Pradeep Agrawal* wrote: try adding in https://github.com/apache/ranger/blob/master/distro/src/main/assembly/plugin-presto.xml make sure you are adding in the directory where you want it. That's what I did. The items are not picked up if the dependency is not present in the pom? On April 5th, 2020, 6:09 p.m. UTC, *Madhan Neethiraj* wrote: ranger-presto-plugin-shim/pom.xml <https://reviews.apache.org/r/72272/diff/2/?file=2215820#file2215820line134> (Diff revision 2) 134 The changes in this module don't seem to refer protobuf-java library contents directly. Is it necessary to explicitly add this dependency? On April 10th, 2020, 8:53 a.m. UTC, *Bolke de Bruin* wrote: Idem as above. How to get it in the assembly? It seems not to be included otherwise (struggling with that overall btw) On April 15th, 2020, 3:04 p.m. UTC, *Pradeep Agrawal* wrote: try adding in https://github.com/apache/ranger/blob/master/distro/src/main/assembly/plugin-presto.xml make sure you are adding in the directory where you want it. Idem as above. - Bolke On April 21st, 2020, 11:43 a.m. UTC, Bolke de Bruin wrote: Review request for ranger, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, and Ramesh Mani. By Bolke de Bruin. *Updated April 21, 2020, 11:43 a.m.* *Bugs: * https://issues.apache.org/jira/browse/RANGER-2754 <https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/RANGER-2754> *Repository: * ranger Description Upgrade and improve Presto plugin - Presto SQL 331 has changed its security API and has Row level / column masking functionality - Upgraded Hadoop dependency to 3.1.3 (from 3.1.1) due to improved security handling - New features like session properties and system properties Testing - New Unit tests added - Tested locally in production Diffs - agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json (56a8f5ac0) - distro/src/main/assembly/plugin-presto.xml (d2075bfe7) - plugin-presto/pom.xml (b63f7dede) - plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java (3ab63f590) - plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerAdminClientImpl.java (PRE-CREATION) - plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControlTest.java (PRE-CREATION) - plugin-presto/src/test/resources/log4j.properties (PRE-CREATION) - plugin-presto/src/test/resources/presto-policies.json (PRE-CREATION) - plugin-presto/src/test/resources/ranger-presto-security.xml (PRE-CREATION) - pom.xml (992c3654e) - ranger-presto-plugin-shim/pom.xml (d8ff88d0f) - ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerConfig.java (67b0d2434) - ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java (e89f646e1) View Diff <https://reviews.apache.org/r/72272/diff/10/>
[jira] [Commented] (RANGER-2754) Update presto dependency and implement row/column level security
[ https://issues.apache.org/jira/browse/RANGER-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17088615#comment-17088615 ] Bolke de Bruin commented on RANGER-2754: Wildcard is set to true now for all items in the row/column policies. > Update presto dependency and implement row/column level security > > > Key: RANGER-2754 > URL: https://issues.apache.org/jira/browse/RANGER-2754 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: master > Reporter: Bolke de Bruin >Assignee: Bolke de Bruin >Priority: Major > Attachments: > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > RANGER-2754-v2.patch, RANGER-2754.patch > > > 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped > working for versions > ~321. > 2. Presto master now has row/column level security support -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 72272: Upgrade and improve Presto plugin
> On April 5, 2020, 6:09 p.m., Madhan Neethiraj wrote: > > agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json > > Lines 96 (patched) > > <https://reviews.apache.org/r/72272/diff/2/?file=2215810#file2215810line96> > > > > To update existing Ranger instances with these service-def changes, > > consider adding a Java patch similar to > > PatchForAtlasToAddEntityLabelAndBusinessMetadata_J10034.java that handles > > recent updates to Atlas service-def. > > > > This can be taken up in a subsequent commit. > > Pradeep Agrawal wrote: > Created RANGER-2795 to track this separately. Thx. Will pick it up after merging this. > On April 5, 2020, 6:09 p.m., Madhan Neethiraj wrote: > > ranger-presto-plugin-shim/pom.xml > > Lines 93 (patched) > > <https://reviews.apache.org/r/72272/diff/2/?file=2215820#file2215820line93> > > > > The changes in this module don't seem to refer hadoop-hdfs library > > contents directly. Is it necessary to explicitly add this dependency? > > Bolke de Bruin wrote: > I'm not getting it into the assembly otherwise for auditing purposes? > please advise. > > Pradeep Agrawal wrote: > try adding in > https://github.com/apache/ranger/blob/master/distro/src/main/assembly/plugin-presto.xml > make sure you are adding in the directory where you want it. That's what I did. The items are not picked up if the dependency is not present in the pom? > On April 5, 2020, 6:09 p.m., Madhan Neethiraj wrote: > > ranger-presto-plugin-shim/pom.xml > > Lines 134 (patched) > > <https://reviews.apache.org/r/72272/diff/2/?file=2215820#file2215820line134> > > > > The changes in this module don't seem to refer protobuf-java library > > contents directly. Is it necessary to explicitly add this dependency? > > Bolke de Bruin wrote: > Idem as above. How to get it in the assembly? It seems not to be included > otherwise (struggling with that overall btw) > > Pradeep Agrawal wrote: > try adding in > https://github.com/apache/ranger/blob/master/distro/src/main/assembly/plugin-presto.xml > make sure you are adding in the directory where you want it. Idem as above. - Bolke ------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72272/#review220223 --- On April 21, 2020, 11:43 a.m., Bolke de Bruin wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/72272/ > --- > > (Updated April 21, 2020, 11:43 a.m.) > > > Review request for ranger, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, > and Ramesh Mani. > > > Bugs: https://issues.apache.org/jira/browse/RANGER-2754 > > https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/RANGER-2754 > > > Repository: ranger > > > Description > --- > > Upgrade and improve Presto plugin > - Presto SQL 331 has changed its security API and has Row level / column > masking functionality > - Upgraded Hadoop dependency to 3.1.3 (from 3.1.1) due to improved security > handling > - New features like session properties and system properties > > > Diffs > - > > agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json > 56a8f5ac0 > distro/src/main/assembly/plugin-presto.xml d2075bfe7 > plugin-presto/pom.xml b63f7dede > > plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java > 3ab63f590 > > plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerAdminClientImpl.java > PRE-CREATION > > plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControlTest.java > PRE-CREATION > plugin-presto/src/test/resources/log4j.properties PRE-CREATION > plugin-presto/src/test/resources/presto-policies.json PRE-CREATION > plugin-presto/src/test/resources/ranger-presto-security.xml PRE-CREATION > pom.xml 992c3654e > ranger-presto-plugin-shim/pom.xml d8ff88d0f > > ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerConfig.java > 67b0d2434 > > ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java > e89f646e1 > > > Diff: https://reviews.apache.org/r/72272/diff/10/ > > > Testing > --- > > - New Unit tests added > - Tested locally in production > > > Thanks, > > Bolke de Bruin > >
Re: Review Request 72272: Upgrade and improve Presto plugin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72272/ --- (Updated April 21, 2020, 11:43 a.m.) Review request for ranger, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, and Ramesh Mani. Changes --- Make row/column matchers user wildcards Bugs: https://issues.apache.org/jira/browse/RANGER-2754 https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/RANGER-2754 Repository: ranger Description --- Upgrade and improve Presto plugin - Presto SQL 331 has changed its security API and has Row level / column masking functionality - Upgraded Hadoop dependency to 3.1.3 (from 3.1.1) due to improved security handling - New features like session properties and system properties Diffs (updated) - agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json 56a8f5ac0 distro/src/main/assembly/plugin-presto.xml d2075bfe7 plugin-presto/pom.xml b63f7dede plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java 3ab63f590 plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerAdminClientImpl.java PRE-CREATION plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControlTest.java PRE-CREATION plugin-presto/src/test/resources/log4j.properties PRE-CREATION plugin-presto/src/test/resources/presto-policies.json PRE-CREATION plugin-presto/src/test/resources/ranger-presto-security.xml PRE-CREATION pom.xml 992c3654e ranger-presto-plugin-shim/pom.xml d8ff88d0f ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerConfig.java 67b0d2434 ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java e89f646e1 Diff: https://reviews.apache.org/r/72272/diff/10/ Changes: https://reviews.apache.org/r/72272/diff/9-10/ Testing --- - New Unit tests added - Tested locally in production Thanks, Bolke de Bruin
[jira] [Commented] (RANGER-2754) Update presto dependency and implement row/column level security
[ https://issues.apache.org/jira/browse/RANGER-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17088597#comment-17088597 ] Bolke de Bruin commented on RANGER-2754: [~ppanda-beta] updated the service definition for RowFiltering and Column Masking > Update presto dependency and implement row/column level security > > > Key: RANGER-2754 > URL: https://issues.apache.org/jira/browse/RANGER-2754 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: master > Reporter: Bolke de Bruin >Assignee: Bolke de Bruin >Priority: Major > Attachments: > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > RANGER-2754-v2.patch, RANGER-2754.patch > > > 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped > working for versions > ~321. > 2. Presto master now has row/column level security support -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (RANGER-2754) Update presto dependency and implement row/column level security
[ https://issues.apache.org/jira/browse/RANGER-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated RANGER-2754: --- Attachment: 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch > Update presto dependency and implement row/column level security > > > Key: RANGER-2754 > URL: https://issues.apache.org/jira/browse/RANGER-2754 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: master > Reporter: Bolke de Bruin > Assignee: Bolke de Bruin >Priority: Major > Attachments: > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > RANGER-2754-v2.patch, RANGER-2754.patch > > > 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped > working for versions > ~321. > 2. Presto master now has row/column level security support -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2754) Update presto dependency and implement row/column level security
[
https://issues.apache.org/jira/browse/RANGER-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17088528#comment-17088528
]
Bolke de Bruin commented on RANGER-2754:
[~ppanda-beta] I think the configuration is correct. wildcard=true and
ignorecase=true in the service definition. This results in
{code}
// If optWildcard is true
// If ('?' found or non-contiguous '*'s found in policyValue)
// needWildcardMatch = true
// End
//
// wildcardStartIdx is set to index of first '*' in
policyValue or -1 if '*' is not found in policyValue, and
// wildcardEndIdx is set to index of last '*' in
policyValue or -1 if '*' is not found in policyValue
// Else
// needWildcardMatch is set to false
// End
if (needWildcardMatch) { // test?, test*a*, test*a*b, *test*a
ret = optIgnoreCase ? new
CaseInsensitiveWildcardMatcher(policyValue) : new
CaseSensitiveWildcardMatcher(policyValue);
} else if (wildcardStartIdx == -1) { // test, testa, testab
ret = optIgnoreCase ? new
CaseInsensitiveStringMatcher(policyValue) : new
CaseSensitiveStringMatcher(policyValue);
{code}
So I think your value is probably incorrect.
> Update presto dependency and implement row/column level security
>
>
> Key: RANGER-2754
> URL: https://issues.apache.org/jira/browse/RANGER-2754
> Project: Ranger
> Issue Type: Improvement
> Components: plugins
>Affects Versions: master
>Reporter: Bolke de Bruin
>Assignee: Bolke de Bruin
>Priority: Major
> Attachments:
> 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch,
> 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch,
> 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch,
> 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch,
> 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch,
> 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch,
> 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch,
> 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch,
> RANGER-2754-v2.patch, RANGER-2754.patch
>
>
> 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped
> working for versions > ~321.
> 2. Presto master now has row/column level security support
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (RANGER-2754) Update presto dependency and implement row/column level security
[ https://issues.apache.org/jira/browse/RANGER-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17088384#comment-17088384 ] Bolke de Bruin commented on RANGER-2754: Ah shoot, I forgot about that one. Will have a look. > Update presto dependency and implement row/column level security > > > Key: RANGER-2754 > URL: https://issues.apache.org/jira/browse/RANGER-2754 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: master > Reporter: Bolke de Bruin >Assignee: Bolke de Bruin >Priority: Major > Attachments: > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > RANGER-2754-v2.patch, RANGER-2754.patch > > > 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped > working for versions > ~321. > 2. Presto master now has row/column level security support -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2754) Update presto dependency and implement row/column level security
[ https://issues.apache.org/jira/browse/RANGER-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17088326#comment-17088326 ] Bolke de Bruin commented on RANGER-2754: If java maintains binary compatibility then it should not be an issue. But why don't you try it? > Update presto dependency and implement row/column level security > > > Key: RANGER-2754 > URL: https://issues.apache.org/jira/browse/RANGER-2754 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: master >Reporter: Bolke de Bruin >Assignee: Bolke de Bruin >Priority: Major > Attachments: > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > RANGER-2754-v2.patch, RANGER-2754.patch > > > 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped > working for versions > ~321. > 2. Presto master now has row/column level security support -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (RANGER-2754) Update presto dependency and implement row/column level security
[ https://issues.apache.org/jira/browse/RANGER-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated RANGER-2754: --- Attachment: 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch > Update presto dependency and implement row/column level security > > > Key: RANGER-2754 > URL: https://issues.apache.org/jira/browse/RANGER-2754 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: master > Reporter: Bolke de Bruin > Assignee: Bolke de Bruin >Priority: Major > Attachments: > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > RANGER-2754-v2.patch, RANGER-2754.patch > > > 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped > working for versions > ~321. > 2. Presto master now has row/column level security support -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2754) Update presto dependency and implement row/column level security
[ https://issues.apache.org/jira/browse/RANGER-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17087865#comment-17087865 ] Bolke de Bruin commented on RANGER-2754: [~ppand-beta] updated. > Update presto dependency and implement row/column level security > > > Key: RANGER-2754 > URL: https://issues.apache.org/jira/browse/RANGER-2754 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: master > Reporter: Bolke de Bruin >Assignee: Bolke de Bruin >Priority: Major > Attachments: > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > RANGER-2754-v2.patch, RANGER-2754.patch > > > 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped > working for versions > ~321. > 2. Presto master now has row/column level security support -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 72272: Upgrade and improve Presto plugin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72272/ --- (Updated April 20, 2020, 3:46 p.m.) Review request for ranger, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, and Ramesh Mani. Changes --- Fixed some issues Bugs: https://issues.apache.org/jira/browse/RANGER-2754 https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/RANGER-2754 Repository: ranger Description --- Upgrade and improve Presto plugin - Presto SQL 331 has changed its security API and has Row level / column masking functionality - Upgraded Hadoop dependency to 3.1.3 (from 3.1.1) due to improved security handling - New features like session properties and system properties Diffs (updated) - agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json 56a8f5ac0 distro/src/main/assembly/plugin-presto.xml d2075bfe7 plugin-presto/pom.xml b63f7dede plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java 3ab63f590 plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerAdminClientImpl.java PRE-CREATION plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControlTest.java PRE-CREATION plugin-presto/src/test/resources/log4j.properties PRE-CREATION plugin-presto/src/test/resources/presto-policies.json PRE-CREATION plugin-presto/src/test/resources/ranger-presto-security.xml PRE-CREATION pom.xml b62d9b663 ranger-presto-plugin-shim/pom.xml d8ff88d0f ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerConfig.java 67b0d2434 ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java e89f646e1 Diff: https://reviews.apache.org/r/72272/diff/9/ Changes: https://reviews.apache.org/r/72272/diff/8-9/ Testing --- - New Unit tests added - Tested locally in production Thanks, Bolke de Bruin
[jira] [Commented] (RANGER-2754) Update presto dependency and implement row/column level security
[
https://issues.apache.org/jira/browse/RANGER-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17087797#comment-17087797
]
Bolke de Bruin commented on RANGER-2754:
Thanks for reporting. I'll fix that tonight. Type is known to ranger by using
{type} which is Presto specific
> Update presto dependency and implement row/column level security
>
>
> Key: RANGER-2754
> URL: https://issues.apache.org/jira/browse/RANGER-2754
> Project: Ranger
> Issue Type: Improvement
> Components: plugins
>Affects Versions: master
>Reporter: Bolke de Bruin
>Assignee: Bolke de Bruin
>Priority: Major
> Attachments:
> 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch,
> 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch,
> 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch,
> 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch,
> 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch,
> 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch,
> 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch,
> RANGER-2754-v2.patch, RANGER-2754.patch
>
>
> 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped
> working for versions > ~321.
> 2. Presto master now has row/column level security support
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (RANGER-2754) Update presto dependency and implement row/column level security
[ https://issues.apache.org/jira/browse/RANGER-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17087683#comment-17087683 ] Bolke de Bruin commented on RANGER-2754: Did you actually encounter the npe? > Update presto dependency and implement row/column level security > > > Key: RANGER-2754 > URL: https://issues.apache.org/jira/browse/RANGER-2754 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: master > Reporter: Bolke de Bruin >Assignee: Bolke de Bruin >Priority: Major > Attachments: > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > RANGER-2754-v2.patch, RANGER-2754.patch > > > 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped > working for versions > ~321. > 2. Presto master now has row/column level security support -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2754) Update presto dependency and implement row/column level security
[ https://issues.apache.org/jira/browse/RANGER-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17087465#comment-17087465 ] Bolke de Bruin commented on RANGER-2754: Latest patch works. Your steps should work, ensure you have the latest service definition applied. That only works on a clean setup (e.g. no previous enabled presto service in ranger). The upgrade script i will create after this patch is accepted. > Update presto dependency and implement row/column level security > > > Key: RANGER-2754 > URL: https://issues.apache.org/jira/browse/RANGER-2754 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: master > Reporter: Bolke de Bruin >Assignee: Bolke de Bruin >Priority: Major > Attachments: > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > RANGER-2754-v2.patch, RANGER-2754.patch > > > 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped > working for versions > ~321. > 2. Presto master now has row/column level security support -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2395) Add presto plugin
[ https://issues.apache.org/jira/browse/RANGER-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17084087#comment-17084087 ] Bolke de Bruin commented on RANGER-2395: That will not work. The current plugin does not support presto >317 due to an api change on The side of presto > Add presto plugin > - > > Key: RANGER-2395 > URL: https://issues.apache.org/jira/browse/RANGER-2395 > Project: Ranger > Issue Type: Improvement > Components: plugins > Reporter: Bolke de Bruin >Assignee: Bolke de Bruin >Priority: Major > Fix For: 2.0.0 > > Attachments: 0001-Add-Presto-plugin.patch > > Time Spent: 7.5h > Remaining Estimate: 0h > > Presto (or PrestoDB) is an open source, distributed SQL query engine, > designed from the ground up for fast analytic queries against data of any > size. It supports both non-relational sources, such as the Hadoop Distributed > File System (HDFS), [Amazon S3|https://aws.amazon.com/s3/], Cassandra, > MongoDB, and [HBase|https://aws.amazon.com/emr/details/hbase/], and > relational data sources such as MySQL, PostgreSQL, [Amazon > Redshift|https://aws.amazon.com/redshift/], Microsoft SQL Server, and > Teradata. > This is to track a Ranger plugin for Presto -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2754) Update presto dependency and implement row/column level security
[ https://issues.apache.org/jira/browse/RANGER-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17084085#comment-17084085 ] Bolke de Bruin commented on RANGER-2754: I suggest building master with this patch and push the service definition into Ranger 1.2 and start presto with the plugin you built. > Update presto dependency and implement row/column level security > > > Key: RANGER-2754 > URL: https://issues.apache.org/jira/browse/RANGER-2754 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: master > Reporter: Bolke de Bruin >Assignee: Bolke de Bruin >Priority: Major > Attachments: > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > RANGER-2754-v2.patch, RANGER-2754.patch > > > 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped > working for versions > ~321. > 2. Presto master now has row/column level security support -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (RANGER-2754) Update presto dependency and implement row/column level security
[ https://issues.apache.org/jira/browse/RANGER-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated RANGER-2754: --- Attachment: 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch > Update presto dependency and implement row/column level security > > > Key: RANGER-2754 > URL: https://issues.apache.org/jira/browse/RANGER-2754 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: master > Reporter: Bolke de Bruin > Assignee: Bolke de Bruin >Priority: Major > Attachments: > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > RANGER-2754-v2.patch, RANGER-2754.patch > > > 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped > working for versions > ~321. > 2. Presto master now has row/column level security support -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 72272: Upgrade and improve Presto plugin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72272/ --- (Updated April 15, 2020, 6:49 a.m.) Review request for ranger, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, and Ramesh Mani. Changes --- Removed 3.1.3 requirement due to guava issues between hive and hadoop (*sigh*) Bugs: https://issues.apache.org/jira/browse/RANGER-2754 https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/RANGER-2754 Repository: ranger Description --- Upgrade and improve Presto plugin - Presto SQL 331 has changed its security API and has Row level / column masking functionality - Upgraded Hadoop dependency to 3.1.3 (from 3.1.1) due to improved security handling - New features like session properties and system properties Diffs (updated) - agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json 56a8f5ac0 distro/src/main/assembly/plugin-presto.xml d2075bfe7 plugin-presto/pom.xml b63f7dede plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java 3ab63f590 plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerAdminClientImpl.java PRE-CREATION plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControlTest.java PRE-CREATION plugin-presto/src/test/resources/log4j.properties PRE-CREATION plugin-presto/src/test/resources/presto-policies.json PRE-CREATION plugin-presto/src/test/resources/ranger-presto-security.xml PRE-CREATION pom.xml b62d9b663 ranger-presto-plugin-shim/pom.xml d8ff88d0f ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerConfig.java 67b0d2434 ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java e89f646e1 Diff: https://reviews.apache.org/r/72272/diff/8/ Changes: https://reviews.apache.org/r/72272/diff/7-8/ Testing --- - New Unit tests added - Tested locally in production Thanks, Bolke de Bruin
Re: Review Request 72272: Upgrade and improve Presto plugin
> On April 13, 2020, 10:16 p.m., Madhan Neethiraj wrote: > > Unit tests fail with this patch - see below for details. This is likely > > caused by change of Hadoop version in this patch (3.1.1 => 3.1.3). Please > > review and update. > > > > > > > > mvn clean package > > ... > > > > [ERROR] Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: > > 0.606 s <<< FAILURE! - in > > org.apache.ranger.services.hive.HIVERangerAuthorizerTest > > [ERROR] org.apache.ranger.services.hive.HIVERangerAuthorizerTest Time > > elapsed: 0.606 s <<< ERROR! > > java.lang.NoSuchMethodError: > > com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V > > at > > org.apache.ranger.services.hive.HIVERangerAuthorizerTest.setup(HIVERangerAuthorizerTest.java:73) > > > > [ERROR] org.apache.ranger.services.hive.HIVERangerAuthorizerTest Time > > elapsed: 0.606 s <<< ERROR! > > java.lang.NullPointerException > > at > > org.apache.ranger.services.hive.HIVERangerAuthorizerTest.cleanup(HIVERangerAuthorizerTest.java:150) > > > > [INFO] > > [INFO] Results: > > [INFO] > > [ERROR] Errors: > > [ERROR] > > org.apache.ranger.services.hive.HIVERangerAuthorizerTest.org.apache.ranger.services.hive.HIVERangerAuthorizerTest > > [ERROR] Run 1: HIVERangerAuthorizerTest.setup:73 » NoSuchMethod > > com.google.common.base.Precon... > > [ERROR] Run 2: HIVERangerAuthorizerTest.cleanup:150 NullPointer > > [INFO] > > [INFO] > > [ERROR] Tests run: 2, Failures: 0, Errors: 1, Skipped: 0 > > ... > > [INFO] HBase Security Plugin .. SUCCESS [ > > 7.327 s] > > [INFO] Hdfs Security Plugin ... SUCCESS [ > > 13.133 s] > > [INFO] Hive Security Plugin ... FAILURE [ > > 9.044 s] > > [INFO] Knox Security Plugin Shim .. SKIPPED > > ... > > [INFO] Apache Ranger Distribution . SKIPPED > > [INFO] > > > > [INFO] BUILD FAILURE > > [INFO] > > > > [INFO] Total time: 01:14 min > > [INFO] Finished at: 2020-04-13T15:08:52-07:00 > > [INFO] > > > > [ERROR] Failed to execute goal > > org.apache.maven.plugins:maven-surefire-plugin:2.21.0:test (default-test) > > on project ranger-hive-plugin: There are test failures. I have been able to reproduce it, although it works in the idea. Tests run: 27, Failures: 0, Errors: 0, Skipped: 3, Time elapsed: 30.037 s - in org.apache.ranger.services.hive.HIVERangerAuthorizerTest - Bolke --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72272/#review220301 --- On April 13, 2020, 11:39 a.m., Bolke de Bruin wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/72272/ > --- > > (Updated April 13, 2020, 11:39 a.m.) > > > Review request for ranger, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, > and Ramesh Mani. > > > Bugs: https://issues.apache.org/jira/browse/RANGER-2754 > > https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/RANGER-2754 > > > Repository: ranger > > > Description > --- > > Upgrade and improve Presto plugin > - Presto SQL 331 has changed its security API and has Row level / column > masking functionality > - Upgraded Hadoop dependency to 3.1.3 (from 3.1.1) due to improved security > handling > - New features like session properties and system properties > > > Diffs > - > > agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json > 56a8f5ac0 > distro/src/main/assembly/plugin-presto.xml d2075bfe7 > plugin-presto/pom.xml b63f7dede > > plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java > 3ab63f590 > > plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerAdminClientImpl.java > PRE-CREATION > > plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerS
[jira] [Commented] (RANGER-2395) Add presto plugin
[ https://issues.apache.org/jira/browse/RANGER-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17082334#comment-17082334 ] Bolke de Bruin commented on RANGER-2395: This plugin is for PrestoSql. I haven't checked PrestoDB. If it exposes the same api it could be just a matter of implementing the interface. > Add presto plugin > - > > Key: RANGER-2395 > URL: https://issues.apache.org/jira/browse/RANGER-2395 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: Bolke de Bruin >Assignee: Bolke de Bruin >Priority: Major > Fix For: 2.0.0 > > Attachments: 0001-Add-Presto-plugin.patch > > Time Spent: 7.5h > Remaining Estimate: 0h > > Presto (or PrestoDB) is an open source, distributed SQL query engine, > designed from the ground up for fast analytic queries against data of any > size. It supports both non-relational sources, such as the Hadoop Distributed > File System (HDFS), [Amazon S3|https://aws.amazon.com/s3/], Cassandra, > MongoDB, and [HBase|https://aws.amazon.com/emr/details/hbase/], and > relational data sources such as MySQL, PostgreSQL, [Amazon > Redshift|https://aws.amazon.com/redshift/], Microsoft SQL Server, and > Teradata. > This is to track a Ranger plugin for Presto -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 72272: Upgrade and improve Presto plugin
> On April 13, 2020, 4:40 a.m., Madhan Neethiraj wrote:
> > plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java
> > Lines 206 (patched)
> > <https://reviews.apache.org/r/72272/diff/6/?file=2217408#file2217408line215>
> >
> > None of the mask-types have token {type}; what is the use case to
> > replace this token in the transformer?
Presto requires types to remain the same when masking (e.g. an varchar should
remain a varchar). This token gives the flexibility to cast to the right {type}
when masking as that will contain the original type.
> On April 13, 2020, 4:40 a.m., Madhan Neethiraj wrote:
> > plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java
> > Line 140 (original), 307 (patched)
> > <https://reviews.apache.org/r/72272/diff/6/?file=2217408#file2217408line317>
> >
> > Are info level logs here and rest of this class intentional? If not,
> > consider changing to debug level.
Good point, will fix that.
- Bolke
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72272/#review220292
-----------
On April 13, 2020, 11:39 a.m., Bolke de Bruin wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72272/
> ---
>
> (Updated April 13, 2020, 11:39 a.m.)
>
>
> Review request for ranger, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal,
> and Ramesh Mani.
>
>
> Bugs: https://issues.apache.org/jira/browse/RANGER-2754
>
> https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/RANGER-2754
>
>
> Repository: ranger
>
>
> Description
> ---
>
> Upgrade and improve Presto plugin
> - Presto SQL 331 has changed its security API and has Row level / column
> masking functionality
> - Upgraded Hadoop dependency to 3.1.3 (from 3.1.1) due to improved security
> handling
> - New features like session properties and system properties
>
>
> Diffs
> -
>
> agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json
> 56a8f5ac0
> distro/src/main/assembly/plugin-presto.xml d2075bfe7
> plugin-presto/pom.xml b63f7dede
>
> plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java
> 3ab63f590
>
> plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerAdminClientImpl.java
> PRE-CREATION
>
> plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControlTest.java
> PRE-CREATION
> plugin-presto/src/test/resources/log4j.properties PRE-CREATION
> plugin-presto/src/test/resources/presto-policies.json PRE-CREATION
> plugin-presto/src/test/resources/ranger-presto-security.xml PRE-CREATION
> pom.xml b62d9b663
> ranger-presto-plugin-shim/pom.xml d8ff88d0f
>
> ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerConfig.java
> 67b0d2434
>
> ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java
> e89f646e1
>
>
> Diff: https://reviews.apache.org/r/72272/diff/7/
>
>
> Testing
> ---
>
> - New Unit tests added
> - Tested locally in production
>
>
> Thanks,
>
> Bolke de Bruin
>
>
[jira] [Updated] (RANGER-2754) Update presto dependency and implement row/column level security
[ https://issues.apache.org/jira/browse/RANGER-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated RANGER-2754: --- Attachment: 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch > Update presto dependency and implement row/column level security > > > Key: RANGER-2754 > URL: https://issues.apache.org/jira/browse/RANGER-2754 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: master > Reporter: Bolke de Bruin > Assignee: Bolke de Bruin >Priority: Major > Attachments: > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > RANGER-2754-v2.patch, RANGER-2754.patch > > > 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped > working for versions > ~321. > 2. Presto master now has row/column level security support -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 72272: Upgrade and improve Presto plugin
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72272/
---
(Updated April 13, 2020, 11:39 a.m.)
Review request for ranger, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, and
Ramesh Mani.
Changes
---
- test for {type} and {col} replacement
- small bugfixes
Bugs: https://issues.apache.org/jira/browse/RANGER-2754
https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/RANGER-2754
Repository: ranger
Description
---
Upgrade and improve Presto plugin
- Presto SQL 331 has changed its security API and has Row level / column
masking functionality
- Upgraded Hadoop dependency to 3.1.3 (from 3.1.1) due to improved security
handling
- New features like session properties and system properties
Diffs (updated)
-
agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json
56a8f5ac0
distro/src/main/assembly/plugin-presto.xml d2075bfe7
plugin-presto/pom.xml b63f7dede
plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java
3ab63f590
plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerAdminClientImpl.java
PRE-CREATION
plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControlTest.java
PRE-CREATION
plugin-presto/src/test/resources/log4j.properties PRE-CREATION
plugin-presto/src/test/resources/presto-policies.json PRE-CREATION
plugin-presto/src/test/resources/ranger-presto-security.xml PRE-CREATION
pom.xml b62d9b663
ranger-presto-plugin-shim/pom.xml d8ff88d0f
ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerConfig.java
67b0d2434
ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java
e89f646e1
Diff: https://reviews.apache.org/r/72272/diff/7/
Changes: https://reviews.apache.org/r/72272/diff/6-7/
Testing
---
- New Unit tests added
- Tested locally in production
Thanks,
Bolke de Bruin
Re: Review Request 72272: Upgrade and improve Presto plugin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72272/ --- (Updated April 10, 2020, 6 p.m.) Review request for ranger, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, and Ramesh Mani. Changes --- make sure not to return null for filter query owners Bugs: https://issues.apache.org/jira/browse/RANGER-2754 https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/RANGER-2754 Repository: ranger Description --- Upgrade and improve Presto plugin - Presto SQL 331 has changed its security API and has Row level / column masking functionality - Upgraded Hadoop dependency to 3.1.3 (from 3.1.1) due to improved security handling - New features like session properties and system properties Diffs (updated) - agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json 56a8f5ac0 distro/src/main/assembly/plugin-presto.xml d2075bfe7 plugin-presto/pom.xml b63f7dede plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java 3ab63f590 plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerAdminClientImpl.java PRE-CREATION plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControlTest.java PRE-CREATION plugin-presto/src/test/resources/log4j.properties PRE-CREATION plugin-presto/src/test/resources/presto-policies.json PRE-CREATION plugin-presto/src/test/resources/ranger-presto-security.xml PRE-CREATION pom.xml b62d9b663 ranger-presto-plugin-shim/pom.xml d8ff88d0f ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerConfig.java 67b0d2434 ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java e89f646e1 Diff: https://reviews.apache.org/r/72272/diff/6/ Changes: https://reviews.apache.org/r/72272/diff/5-6/ Testing --- - New Unit tests added - Tested locally in production Thanks, Bolke de Bruin
[jira] [Updated] (RANGER-2754) Update presto dependency and implement row/column level security
[ https://issues.apache.org/jira/browse/RANGER-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated RANGER-2754: --- Attachment: 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch > Update presto dependency and implement row/column level security > > > Key: RANGER-2754 > URL: https://issues.apache.org/jira/browse/RANGER-2754 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: master > Reporter: Bolke de Bruin > Assignee: Bolke de Bruin >Priority: Major > Attachments: > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > RANGER-2754-v2.patch, RANGER-2754.patch > > > 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped > working for versions > ~321. > 2. Presto master now has row/column level security support -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (RANGER-2754) Update presto dependency and implement row/column level security
[ https://issues.apache.org/jira/browse/RANGER-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated RANGER-2754: --- Attachment: 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch > Update presto dependency and implement row/column level security > > > Key: RANGER-2754 > URL: https://issues.apache.org/jira/browse/RANGER-2754 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: master > Reporter: Bolke de Bruin > Assignee: Bolke de Bruin >Priority: Major > Attachments: > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > RANGER-2754-v2.patch, RANGER-2754.patch > > > 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped > working for versions > ~321. > 2. Presto master now has row/column level security support -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 72272: Upgrade and improve Presto plugin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72272/ --- (Updated April 10, 2020, 5:16 p.m.) Review request for ranger, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, and Ramesh Mani. Changes --- Includes tests for filter(Catalogs/Schema/Tables) Bugs: https://issues.apache.org/jira/browse/RANGER-2754 https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/RANGER-2754 Repository: ranger Description --- Upgrade and improve Presto plugin - Presto SQL 331 has changed its security API and has Row level / column masking functionality - Upgraded Hadoop dependency to 3.1.3 (from 3.1.1) due to improved security handling - New features like session properties and system properties Diffs (updated) - agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json 56a8f5ac0 distro/src/main/assembly/plugin-presto.xml d2075bfe7 plugin-presto/pom.xml b63f7dede plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java 3ab63f590 plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerAdminClientImpl.java PRE-CREATION plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControlTest.java PRE-CREATION plugin-presto/src/test/resources/log4j.properties PRE-CREATION plugin-presto/src/test/resources/presto-policies.json PRE-CREATION plugin-presto/src/test/resources/ranger-presto-security.xml PRE-CREATION pom.xml b62d9b663 ranger-presto-plugin-shim/pom.xml d8ff88d0f ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerConfig.java 67b0d2434 ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java e89f646e1 Diff: https://reviews.apache.org/r/72272/diff/5/ Changes: https://reviews.apache.org/r/72272/diff/4-5/ Testing --- - New Unit tests added - Tested locally in production Thanks, Bolke de Bruin
Re: Review Request 72272: Upgrade and improve Presto plugin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72272/ --- (Updated April 10, 2020, 9:09 a.m.) Review request for ranger, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, and Ramesh Mani. Changes --- remove unused import Bugs: https://issues.apache.org/jira/browse/RANGER-2754 https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/RANGER-2754 Repository: ranger Description --- Upgrade and improve Presto plugin - Presto SQL 331 has changed its security API and has Row level / column masking functionality - Upgraded Hadoop dependency to 3.1.3 (from 3.1.1) due to improved security handling - New features like session properties and system properties Diffs (updated) - agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json 56a8f5ac0 distro/src/main/assembly/plugin-presto.xml d2075bfe7 plugin-presto/pom.xml b63f7dede plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java 3ab63f590 plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerAdminClientImpl.java PRE-CREATION plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControlTest.java PRE-CREATION plugin-presto/src/test/resources/log4j.properties PRE-CREATION plugin-presto/src/test/resources/presto-policies.json PRE-CREATION plugin-presto/src/test/resources/ranger-presto-security.xml PRE-CREATION pom.xml 22926fd7d ranger-presto-plugin-shim/pom.xml d8ff88d0f ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerConfig.java 67b0d2434 ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java e89f646e1 Diff: https://reviews.apache.org/r/72272/diff/4/ Changes: https://reviews.apache.org/r/72272/diff/3-4/ Testing --- - New Unit tests added - Tested locally in production Thanks, Bolke de Bruin
[jira] [Updated] (RANGER-2754) Update presto dependency and implement row/column level security
[ https://issues.apache.org/jira/browse/RANGER-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated RANGER-2754: --- Attachment: 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch > Update presto dependency and implement row/column level security > > > Key: RANGER-2754 > URL: https://issues.apache.org/jira/browse/RANGER-2754 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: master > Reporter: Bolke de Bruin > Assignee: Bolke de Bruin >Priority: Major > Attachments: > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > RANGER-2754-v2.patch, RANGER-2754.patch > > > 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped > working for versions > ~321. > 2. Presto master now has row/column level security support -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (RANGER-2754) Update presto dependency and implement row/column level security
[ https://issues.apache.org/jira/browse/RANGER-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated RANGER-2754: --- Attachment: 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch > Update presto dependency and implement row/column level security > > > Key: RANGER-2754 > URL: https://issues.apache.org/jira/browse/RANGER-2754 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: master > Reporter: Bolke de Bruin > Assignee: Bolke de Bruin >Priority: Major > Attachments: > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > RANGER-2754-v2.patch, RANGER-2754.patch > > > 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped > working for versions > ~321. > 2. Presto master now has row/column level security support -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2754) Update presto dependency and implement row/column level security
[ https://issues.apache.org/jira/browse/RANGER-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17080363#comment-17080363 ] Bolke de Bruin commented on RANGER-2754: [~liujiayi771] I have included your changes in the latest update to this patch. I hope you don't mind. > Update presto dependency and implement row/column level security > > > Key: RANGER-2754 > URL: https://issues.apache.org/jira/browse/RANGER-2754 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: master >Reporter: Bolke de Bruin >Assignee: Bolke de Bruin >Priority: Major > Attachments: > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > RANGER-2754-v2.patch, RANGER-2754.patch > > > 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped > working for versions > ~321. > 2. Presto master now has row/column level security support -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 72272: Upgrade and improve Presto plugin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72272/ --- (Updated April 10, 2020, 8:55 a.m.) Review request for ranger, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, and Ramesh Mani. Changes --- Addressed issues Bugs: https://issues.apache.org/jira/browse/RANGER-2754 https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/RANGER-2754 Repository: ranger Description --- Upgrade and improve Presto plugin - Presto SQL 331 has changed its security API and has Row level / column masking functionality - Upgraded Hadoop dependency to 3.1.3 (from 3.1.1) due to improved security handling - New features like session properties and system properties Diffs (updated) - agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json 56a8f5ac0 distro/src/main/assembly/plugin-presto.xml d2075bfe7 plugin-presto/pom.xml b63f7dede plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java 3ab63f590 plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerAdminClientImpl.java PRE-CREATION plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControlTest.java PRE-CREATION plugin-presto/src/test/resources/log4j.properties PRE-CREATION plugin-presto/src/test/resources/presto-policies.json PRE-CREATION plugin-presto/src/test/resources/ranger-presto-security.xml PRE-CREATION pom.xml 22926fd7d ranger-presto-plugin-shim/pom.xml d8ff88d0f ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerConfig.java 67b0d2434 ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java e89f646e1 Diff: https://reviews.apache.org/r/72272/diff/3/ Changes: https://reviews.apache.org/r/72272/diff/2-3/ Testing --- - New Unit tests added - Tested locally in production Thanks, Bolke de Bruin
Re: Review Request 72272: Upgrade and improve Presto plugin
> On April 5, 2020, 6:09 p.m., Madhan Neethiraj wrote: > > plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java > > Lines 74 (patched) > > <https://reviews.apache.org/r/72272/diff/2/?file=2215813#file2215813line76> > > > > Since 'useUgi' is set only in the constructor, consider marking this > > as a 'final'. Same applies for 'rangerPlugin' as well. Please review. that won't work for useUgi. Declaring it final thecompiler complains that it cannot set it. > On April 5, 2020, 6:09 p.m., Madhan Neethiraj wrote: > > plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java > > Lines 157 (patched) > > <https://reviews.apache.org/r/72272/diff/2/?file=2215813#file2215813line166> > > > > result.isRowFilterEnabled() already checks if the filter-expr is empty > > or not; so 'StringUtils.isNotEmpty(result.getFilterExpr());' is not needed > > here. Please review. came from the Hive Authorizer. Removed > On April 5, 2020, 6:09 p.m., Madhan Neethiraj wrote: > > ranger-presto-plugin-shim/pom.xml > > Lines 93 (patched) > > <https://reviews.apache.org/r/72272/diff/2/?file=2215820#file2215820line93> > > > > The changes in this module don't seem to refer hadoop-hdfs library > > contents directly. Is it necessary to explicitly add this dependency? I'm not getting it into the assembly otherwise for auditing purposes? please advise. > On April 5, 2020, 6:09 p.m., Madhan Neethiraj wrote: > > ranger-presto-plugin-shim/pom.xml > > Lines 134 (patched) > > <https://reviews.apache.org/r/72272/diff/2/?file=2215820#file2215820line134> > > > > The changes in this module don't seem to refer protobuf-java library > > contents directly. Is it necessary to explicitly add this dependency? Idem as above. How to get it in the assembly? It seems not to be included otherwise (struggling with that overall btw) - Bolke ----------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72272/#review220223 --- On March 30, 2020, 5:20 p.m., Bolke de Bruin wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/72272/ > --- > > (Updated March 30, 2020, 5:20 p.m.) > > > Review request for ranger, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, > and Ramesh Mani. > > > Bugs: https://issues.apache.org/jira/browse/RANGER-2754 > > https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/RANGER-2754 > > > Repository: ranger > > > Description > --- > > Upgrade and improve Presto plugin > - Presto SQL 331 has changed its security API and has Row level / column > masking functionality > - Upgraded Hadoop dependency to 3.1.3 (from 3.1.1) due to improved security > handling > - New features like session properties and system properties > > > Diffs > - > > agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json > 56a8f5ac0 > distro/src/main/assembly/plugin-presto.xml d2075bfe7 > plugin-presto/pom.xml b63f7dede > > plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java > 3ab63f590 > > plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerAdminClientImpl.java > PRE-CREATION > > plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControlTest.java > PRE-CREATION > plugin-presto/src/test/resources/log4j.properties PRE-CREATION > plugin-presto/src/test/resources/presto-policies.json PRE-CREATION > plugin-presto/src/test/resources/ranger-presto-security.xml PRE-CREATION > pom.xml 22926fd7d > ranger-presto-plugin-shim/pom.xml d8ff88d0f > > ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerConfig.java > 67b0d2434 > > ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java > e89f646e1 > > > Diff: https://reviews.apache.org/r/72272/diff/2/ > > > Testing > --- > > - New Unit tests added > - Tested locally in production > > > Thanks, > > Bolke de Bruin > >
[jira] [Commented] (RANGER-2782) Upgrade log4j dependency
[ https://issues.apache.org/jira/browse/RANGER-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17080307#comment-17080307 ] Bolke de Bruin commented on RANGER-2782: Yes working on that. It will be much larger though. > Upgrade log4j dependency > > > Key: RANGER-2782 > URL: https://issues.apache.org/jira/browse/RANGER-2782 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0 > Reporter: Bolke de Bruin >Assignee: Bolke de Bruin >Priority: Blocker > Fix For: 2.1.0 > > Attachments: > 0001-RANGER-2782-Upgrade-log4j-to-a-supported-version.patch > > > The current log4j version in ranger is end of life and contains critical > security Vulnerabilities > CVE-2019-17571 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (RANGER-2787) Upgrade presto version to 331
[ https://issues.apache.org/jira/browse/RANGER-2787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin reopened RANGER-2787: > Upgrade presto version to 331 > - > > Key: RANGER-2787 > URL: https://issues.apache.org/jira/browse/RANGER-2787 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: Jiayi Liu >Priority: Major > Fix For: 2.1 > > Attachments: 0001-RANGER-2787.patch > > > Upgrade presto dependency to version 331 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (RANGER-2787) Upgrade presto version to 331
[ https://issues.apache.org/jira/browse/RANGER-2787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin resolved RANGER-2787. Resolution: Duplicate > Upgrade presto version to 331 > - > > Key: RANGER-2787 > URL: https://issues.apache.org/jira/browse/RANGER-2787 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: Jiayi Liu >Priority: Major > Fix For: 2.1 > > Attachments: 0001-RANGER-2787.patch > > > Upgrade presto dependency to version 331 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (RANGER-2787) Upgrade presto version to 331
[ https://issues.apache.org/jira/browse/RANGER-2787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin resolved RANGER-2787. Resolution: Fixed > Upgrade presto version to 331 > - > > Key: RANGER-2787 > URL: https://issues.apache.org/jira/browse/RANGER-2787 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: Jiayi Liu >Priority: Major > Fix For: 2.1 > > Attachments: 0001-RANGER-2787.patch > > > Upgrade presto dependency to version 331 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (RANGER-2782) Upgrade log4j dependency
[ https://issues.apache.org/jira/browse/RANGER-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated RANGER-2782: --- External issue URL: https://reviews.apache.org/r/72331/ > Upgrade log4j dependency > > > Key: RANGER-2782 > URL: https://issues.apache.org/jira/browse/RANGER-2782 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0 > Reporter: Bolke de Bruin > Assignee: Bolke de Bruin >Priority: Blocker > Fix For: 2.1.0 > > Attachments: > 0001-RANGER-2782-Upgrade-log4j-to-a-supported-version.patch > > > The current log4j version in ranger is end of life and contains critical > security Vulnerabilities > CVE-2019-17571 -- This message was sent by Atlassian Jira (v8.3.4#803005)
Review Request 72331: Upgrade log4j dependency
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72331/ --- Review request for ranger and Madhan Neethiraj. Bugs: RANGER-2782 https://issues.apache.org/jira/browse/RANGER-2782 Repository: ranger Description --- The current log4j version in ranger is end of life and contains critical security Vulnerabilities CVE-2019-17571 Diffs - agents-audit/pom.xml 8ac1edf4f agents-common/pom.xml c78dc5fc2 agents-cred/pom.xml cd1b8f3e3 embeddedwebserver/pom.xml 8574c5721 kms/pom.xml 3bf20fdd4 plugin-schema-registry/pom.xml 6bd2d9766 pom.xml 22926fd7d ranger-examples/sampleapp/pom.xml 494fea3a5 security-admin/pom.xml fc4a20020 tagsync/pom.xml b8340c063 ugsync/pom.xml b1d695af1 unixauthclient/pom.xml fa5d40966 unixauthservice/pom.xml 7cd6aecc2 Diff: https://reviews.apache.org/r/72331/diff/1/ Testing --- - started ranger Thanks, Bolke de Bruin
[jira] [Commented] (RANGER-2702) Upgrade Kafka Version in Ranger to 2.4
[ https://issues.apache.org/jira/browse/RANGER-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077167#comment-17077167 ] Bolke de Bruin commented on RANGER-2702: I know, this leaves the current release (2.0.0) vulnerable as this is only in master at the moment. Ie. I'm suggesting a back port to 2.0 or to release 2.1 > Upgrade Kafka Version in Ranger to 2.4 > -- > > Key: RANGER-2702 > URL: https://issues.apache.org/jira/browse/RANGER-2702 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0, 2.1.0 >Reporter: Ramesh Mani >Assignee: Ramesh Mani >Priority: Major > Labels: security > Fix For: 2.1.0 > > > Upgrade Kafka Version in Ranger to 2.4. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (RANGER-2702) Upgrade Kafka Version in Ranger to 2.4
[ https://issues.apache.org/jira/browse/RANGER-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077167#comment-17077167 ] Bolke de Bruin edited comment on RANGER-2702 at 4/7/20, 12:15 PM: -- I know, this leaves the current release (Ranger 2.0.0) vulnerable as this is only in master at the moment. Ie. I'm suggesting a back port to 2.0 or to release 2.1 was (Author: bolke): I know, this leaves the current release (2.0.0) vulnerable as this is only in master at the moment. Ie. I'm suggesting a back port to 2.0 or to release 2.1 > Upgrade Kafka Version in Ranger to 2.4 > -- > > Key: RANGER-2702 > URL: https://issues.apache.org/jira/browse/RANGER-2702 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0, 2.1.0 >Reporter: Ramesh Mani >Assignee: Ramesh Mani >Priority: Major > Labels: security > Fix For: 2.1.0 > > > Upgrade Kafka Version in Ranger to 2.4. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (RANGER-2782) Upgrade log4j dependency
[ https://issues.apache.org/jira/browse/RANGER-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated RANGER-2782: --- Attachment: 0001-RANGER-2782-Upgrade-log4j-to-a-supported-version.patch > Upgrade log4j dependency > > > Key: RANGER-2782 > URL: https://issues.apache.org/jira/browse/RANGER-2782 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0 > Reporter: Bolke de Bruin > Assignee: Bolke de Bruin >Priority: Blocker > Fix For: 2.1.0 > > Attachments: > 0001-RANGER-2782-Upgrade-log4j-to-a-supported-version.patch > > > The current log4j version in ranger is end of life and contains critical > security Vulnerabilities > CVE-2019-17571 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (RANGER-2782) Upgrade log4j dependency
[ https://issues.apache.org/jira/browse/RANGER-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin reassigned RANGER-2782: -- Assignee: Bolke de Bruin > Upgrade log4j dependency > > > Key: RANGER-2782 > URL: https://issues.apache.org/jira/browse/RANGER-2782 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0 > Reporter: Bolke de Bruin > Assignee: Bolke de Bruin >Priority: Blocker > Fix For: 2.1.0 > > > The current log4j version in ranger is end of life and contains critical > security Vulnerabilities > CVE-2019-17571 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (RANGER-2702) Upgrade Kafka Version in Ranger to 2.4
[ https://issues.apache.org/jira/browse/RANGER-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated RANGER-2702: --- Labels: security (was: ) > Upgrade Kafka Version in Ranger to 2.4 > -- > > Key: RANGER-2702 > URL: https://issues.apache.org/jira/browse/RANGER-2702 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0, 2.1.0 >Reporter: Ramesh Mani >Assignee: Ramesh Mani >Priority: Major > Labels: security > Fix For: 2.1.0 > > > Upgrade Kafka Version in Ranger to 2.4. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (RANGER-2702) Upgrade Kafka Version in Ranger to 2.4
[ https://issues.apache.org/jira/browse/RANGER-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated RANGER-2702: --- Affects Version/s: 2.0.0 > Upgrade Kafka Version in Ranger to 2.4 > -- > > Key: RANGER-2702 > URL: https://issues.apache.org/jira/browse/RANGER-2702 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0, 2.1.0 >Reporter: Ramesh Mani >Assignee: Ramesh Mani >Priority: Major > Fix For: 2.1.0 > > > Upgrade Kafka Version in Ranger to 2.4. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2702) Upgrade Kafka Version in Ranger to 2.4
[ https://issues.apache.org/jira/browse/RANGER-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077161#comment-17077161 ] Bolke de Bruin commented on RANGER-2702: Kafka 2.0.0 contains critical CVEs: CVE-2019-12399, CVE-2018-17196 is it possible to have a back port to 2.0 and have a minor release (2.0.1)? > Upgrade Kafka Version in Ranger to 2.4 > -- > > Key: RANGER-2702 > URL: https://issues.apache.org/jira/browse/RANGER-2702 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.1.0 >Reporter: Ramesh Mani >Assignee: Ramesh Mani >Priority: Major > Fix For: 2.1.0 > > > Upgrade Kafka Version in Ranger to 2.4. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (RANGER-2782) Upgrade log4j dependency
Bolke de Bruin created RANGER-2782: -- Summary: Upgrade log4j dependency Key: RANGER-2782 URL: https://issues.apache.org/jira/browse/RANGER-2782 Project: Ranger Issue Type: Bug Components: Ranger Affects Versions: 2.0.0 Reporter: Bolke de Bruin The current log4j version in ranger is end of life and contains critical security Vulnerabilities CVE-2019-17571 -- This message was sent by Atlassian Jira (v8.3.4#803005)
Fwd: Review Request 72272: Upgrade and improve Presto plugin
friendly ping Thx Bolke Verstuurd vanaf mijn iPad Begin doorgestuurd bericht: > Van: Bolke de Bruin > Datum: 30 maart 2020 om 19:20:10 CEST > Aan: Pradeep Agrawal , Mehul Parikh > , Madhan Neethiraj , Ramesh Mani > , Bolke de Bruin > Onderwerp: Antw: Review Request 72272: Upgrade and improve Presto plugin > Antwoord aan: Bolke de Bruin > > > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/72272/ > > Review request for ranger, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, > and Ramesh Mani. > By Bolke de Bruin. > Updated March 30, 2020, 5:20 p.m. > > Bugs: https://issues.apache.org/jira/browse/RANGER-2754 > Repository: ranger > Description > > Upgrade and improve Presto plugin > - Presto SQL 331 has changed its security API and has Row level / column > masking functionality > - Upgraded Hadoop dependency to 3.1.3 (from 3.1.1) due to improved security > handling > - New features like session properties and system properties > Testing > > New Unit tests added > Tested locally in production > Diffs > > agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json > (56a8f5ac0) > distro/src/main/assembly/plugin-presto.xml (d2075bfe7) > plugin-presto/pom.xml (b63f7dede) > plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java > (3ab63f590) > plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerAdminClientImpl.java > (PRE-CREATION) > plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControlTest.java > (PRE-CREATION) > plugin-presto/src/test/resources/log4j.properties (PRE-CREATION) > plugin-presto/src/test/resources/presto-policies.json (PRE-CREATION) > plugin-presto/src/test/resources/ranger-presto-security.xml (PRE-CREATION) > pom.xml (22926fd7d) > ranger-presto-plugin-shim/pom.xml (d8ff88d0f) > ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerConfig.java > (67b0d2434) > ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java > (e89f646e1) > View Diff
Fwd: Review Request for Presto Plugin Upgrade
Friendly Ping! Sent from my iPhone Begin forwarded message: > From: Bolke de Bruin > Date: 26 March 2020 at 14:45:02 CET > To: dev@ranger.apache.org > Subject: Review Request for Presto Plugin Upgrade > Reply-To: dev@ranger.apache.org > > Hi, > > Presto has upgraded its security API and hence the Ranger plugin is not > compatible anymore. Furthermore I have made many improvements to the plugin > and added row level filtering and column masking. > > Please review: https://reviews.apache.org/r/72272/ > > And https://issues.apache.org/jira/browse/RANGER-2754 > > Thanks > Bolke >
Re: Review Request 72104: ATLAS-3610 Enable logging to STDOUT in Atlas startup scripts
Can this be merged please? Sent from my iPhone > On 26 Mar 2020, at 14:38, Bolke de Bruin wrote: > > > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/72104/ > > Ship it! > > Ship It! > > - Bolke de Bruin > > > On March 26th, 2020, 7:41 a.m. UTC, Mariusz Górski wrote: > > Review request for atlas, Ashutosh Mestry, Bolke de Bruin, Madhan Neethiraj, > Nixon Rodrigues, and Sarath Subramanian. > By Mariusz Górski. > Updated March 26, 2020, 7:41 a.m. > > Repository: atlas > Description > > This patch enables logging to STDOUT from processes spawned through > atlas_start.py or atlas_config.py scripts. For now, even if configured in > log4j, logging to STDOUT was supressed (by redirecting to log files only). > Testing > > The tests were concluded by: > > Building & running Atlas locally with default log4j configuration and > -Dlog.console mvn build option set to true. > Building & running Atlas on Openshift with log4j configuration pushing > everything to STDOUT and -Dlog.console mvn build option set to true. > Running distro tests on localhost > > In both scenarios logs were available both in the .out/.err files in the > Atlas logdir (backwards compatibility), but at the same time I was able to > see them in STDOUT of both running process (local installation) and pod > (openshift deployment). > Diffs > > distro/pom.xml (7159b16cf) > distro/src/bin/atlas_config.py (f09026ff9) > distro/src/bin/atlas_start.py (963faf402) > distro/src/bin/cputil.py (98a9dc36d) > distro/src/conf/atlas-env.sh (c4241e665) > distro/src/test/python/scripts/TestMetadata.py (f1235f747) > View Diff
[jira] [Commented] (RANGER-2754) Update presto dependency and implement row/column level security
[ https://issues.apache.org/jira/browse/RANGER-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17067700#comment-17067700 ] Bolke de Bruin commented on RANGER-2754: Patch ready for review > Update presto dependency and implement row/column level security > > > Key: RANGER-2754 > URL: https://issues.apache.org/jira/browse/RANGER-2754 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: master > Reporter: Bolke de Bruin >Assignee: Bolke de Bruin >Priority: Major > Attachments: > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > RANGER-2754-v2.patch, RANGER-2754.patch > > > 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped > working for versions > ~321. > 2. Presto master now has row/column level security support -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 72272: Upgrade and improve Presto plugin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72272/ --- (Updated March 26, 2020, 2:07 p.m.) Review request for ranger. Bugs: https://issues.apache.org/jira/browse/RANGER-2754 https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/RANGER-2754 Repository: ranger Description --- Upgrade and improve Presto plugin - Presto SQL 331 has changed its security API and has Row level / column masking functionality - Upgraded Hadoop dependency to 3.1.3 (from 3.1.1) due to improved security handling - New features like session properties and system properties Diffs (updated) - agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json 56a8f5ac0 distro/src/main/assembly/plugin-presto.xml d2075bfe7 plugin-presto/pom.xml b63f7dede plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java 3ab63f590 plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerAdminClientImpl.java PRE-CREATION plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControlTest.java PRE-CREATION plugin-presto/src/test/resources/log4j.properties PRE-CREATION plugin-presto/src/test/resources/presto-policies.json PRE-CREATION plugin-presto/src/test/resources/ranger-presto-security.xml PRE-CREATION pom.xml 22926fd7d ranger-presto-plugin-shim/pom.xml d8ff88d0f ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerConfig.java 67b0d2434 ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java e89f646e1 Diff: https://reviews.apache.org/r/72272/diff/2/ Changes: https://reviews.apache.org/r/72272/diff/1-2/ Testing --- - New Unit tests added - Tested locally in production Thanks, Bolke de Bruin
[jira] [Updated] (RANGER-2754) Update presto dependency and implement row/column level security
[ https://issues.apache.org/jira/browse/RANGER-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated RANGER-2754: --- Attachment: RANGER-2754-v2.patch > Update presto dependency and implement row/column level security > > > Key: RANGER-2754 > URL: https://issues.apache.org/jira/browse/RANGER-2754 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: master > Reporter: Bolke de Bruin > Assignee: Bolke de Bruin >Priority: Major > Attachments: > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > RANGER-2754-v2.patch, RANGER-2754.patch > > > 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped > working for versions > ~321. > 2. Presto master now has row/column level security support -- This message was sent by Atlassian Jira (v8.3.4#803005)
Review Request for Presto Plugin Upgrade
Hi, Presto has upgraded its security API and hence the Ranger plugin is not compatible anymore. Furthermore I have made many improvements to the plugin and added row level filtering and column masking. Please review: https://reviews.apache.org/r/72272/ And https://issues.apache.org/jira/browse/RANGER-2754 Thanks Bolke
Review Request 72272: Upgrade and improve Presto plugin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72272/ --- Review request for ranger. Bugs: https://issues.apache.org/jira/browse/RANGER-2754 https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/RANGER-2754 Repository: ranger Description --- Upgrade and improve Presto plugin - Presto SQL 331 has changed its security API and has Row level / column masking functionality - Upgraded Hadoop dependency to 3.1.3 (from 3.1.1) due to improved security handling - New features like session properties and system properties Diffs - agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json 56a8f5ac0 distro/src/main/assembly/plugin-presto.xml d2075bfe7 plugin-presto/conf/ranger-presto-security.xml 9feae81a6 plugin-presto/pom.xml b63f7dede plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java 3ab63f590 plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerAdminClientImpl.java PRE-CREATION plugin-presto/src/test/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControlTest.java PRE-CREATION plugin-presto/src/test/resources/log4j.properties PRE-CREATION plugin-presto/src/test/resources/presto-policies.json PRE-CREATION plugin-presto/src/test/resources/ranger-presto-security.xml PRE-CREATION pom.xml 22926fd7d ranger-presto-plugin-shim/pom.xml d8ff88d0f ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerConfig.java 67b0d2434 ranger-presto-plugin-shim/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java e89f646e1 Diff: https://reviews.apache.org/r/72272/diff/1/ Testing --- - New Unit tests added - Tested locally in production Thanks, Bolke de Bruin
[jira] [Updated] (RANGER-2754) Update presto dependency and implement row/column level security
[ https://issues.apache.org/jira/browse/RANGER-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated RANGER-2754: --- Attachment: RANGER-2754.patch > Update presto dependency and implement row/column level security > > > Key: RANGER-2754 > URL: https://issues.apache.org/jira/browse/RANGER-2754 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: master > Reporter: Bolke de Bruin > Assignee: Bolke de Bruin >Priority: Major > Attachments: > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch, > RANGER-2754.patch > > > 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped > working for versions > ~321. > 2. Presto master now has row/column level security support -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (RANGER-2754) Update presto dependency and implement row/column level security
[ https://issues.apache.org/jira/browse/RANGER-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin reassigned RANGER-2754: -- Assignee: Bolke de Bruin > Update presto dependency and implement row/column level security > > > Key: RANGER-2754 > URL: https://issues.apache.org/jira/browse/RANGER-2754 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: master > Reporter: Bolke de Bruin > Assignee: Bolke de Bruin >Priority: Major > Attachments: > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch > > > 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped > working for versions > ~321. > 2. Presto master now has row/column level security support -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2754) Update presto dependency and implement row/column level security
[ https://issues.apache.org/jira/browse/RANGER-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17061945#comment-17061945 ] Bolke de Bruin commented on RANGER-2754: The attached patch is for testing. It might still need some adjustments. This supports Presto 331 and above and includes row level filtering and column masking support. It requires updating the service definition. > Update presto dependency and implement row/column level security > > > Key: RANGER-2754 > URL: https://issues.apache.org/jira/browse/RANGER-2754 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: master > Reporter: Bolke de Bruin >Priority: Major > Attachments: > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch > > > 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped > working for versions > ~321. > 2. Presto master now has row/column level security support -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (RANGER-2754) Update presto dependency and implement row/column level security
[ https://issues.apache.org/jira/browse/RANGER-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated RANGER-2754: --- Attachment: 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch > Update presto dependency and implement row/column level security > > > Key: RANGER-2754 > URL: https://issues.apache.org/jira/browse/RANGER-2754 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: master > Reporter: Bolke de Bruin >Priority: Major > Attachments: > 0001-RANGER-2754-Upgrade-presto-dependency-and-improve-lo.patch > > > 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped > working for versions > ~321. > 2. Presto master now has row/column level security support -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Failed to create assembly with master branch works
Gotcha. Maybe it's smart then to update the build instructions. I think I got it from there. Cheers Bolke Sent from my iPhone > On 8 Mar 2020, at 17:49, Madhan Neethiraj wrote: > > Bolke, > > "assembly:assembly" should not be used after recent updates to use assembly > plugin version 2.6 (RANGER-837). Please build without this argument. > > Madhan > > On 3/8/20, 3:27 AM, "Bolke de Bruin" wrote: > > >Hi All, > >I am trying to build Ranger from master to update the Presto dependencies. > This always fails with an error while building the assembly with both JDK 8 > and JDK 11 (with a small patch) > >"org.codehaus.plexus.archiver.ArchiverException: Problem reading from > source file in xAR operation ZipFile invalid LOC header (bad signature)” > >The normal action for this is to clean your .m2 directory, which of course > I did. This doesn’t resolve the issue. Accidentally I switched to an older > checkout RANGER-2.0.0-SNAPSHOT and this build finishes fine. > >Anyone aware of this? > >mvn -DskipTests=true clean compile package install assembly:assembly > > >Thanks >Bolke > >
Failed to create assembly with master branch works
Hi All, I am trying to build Ranger from master to update the Presto dependencies. This always fails with an error while building the assembly with both JDK 8 and JDK 11 (with a small patch) "org.codehaus.plexus.archiver.ArchiverException: Problem reading from source file in xAR operation ZipFile invalid LOC header (bad signature)” The normal action for this is to clean your .m2 directory, which of course I did. This doesn’t resolve the issue. Accidentally I switched to an older checkout RANGER-2.0.0-SNAPSHOT and this build finishes fine. Anyone aware of this? mvn -DskipTests=true clean compile package install assembly:assembly Thanks Bolke
[jira] [Commented] (RANGER-2751) SSL enabled Apache Ranger (2.1.0) not working with SSL enabled Presto (Prestosql 310) - Policy synch up not happening
[
https://issues.apache.org/jira/browse/RANGER-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17053348#comment-17053348
]
Bolke de Bruin commented on RANGER-2751:
https://issues.apache.org/jira/browse/RANGER-2611
Check what the config file of the plugin actually says (your ranger-XXX.xml)
> SSL enabled Apache Ranger (2.1.0) not working with SSL enabled Presto
> (Prestosql 310) - Policy synch up not happening
> --
>
> Key: RANGER-2751
> URL: https://issues.apache.org/jira/browse/RANGER-2751
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Affects Versions: 2.1.0
>Reporter: sajai
>Priority: Major
> Fix For: 2.1.0
>
>
> *Facing the below error when trying to integrate Apache Ranger with Prestosql
> (310 version).*
> *Both Ranger and Presto is working independently, but the Presto policies
> from Ranger are not downloading/refreshing. Couldn't find the policies
> downloaded in Ranger web ui in Audits/Plugin tab. Also if we remove SSL from
> Ranger side it starts working fine. Issue is only when SSL is enabled in
> Ranger, then Presto inot working with Ranger,*
> 2020-03-04T07:50:59.600-0600 ERROR Thread-91
> org.apache.ranger.plugin.util.PolicyRefresher
> PolicyRefresher(serviceName=presto-catalogs-dev): failed to refresh policies.
> Will continue to use last known version of policies (-1)
> java.lang.IllegalArgumentException: TrustManager is not specified
> *ranger-2.1.0-SNAPSHOT-admin/install.properties:-*
> db_root_user=root
> db_root_password=Sqlpwd@123
> db_host=localhost
> db_name=ranger
> db_user=rangeradmin
> db_password=Rangerpwd@123
> rangerAdmin_password=Rangerpwd@123
> rangerTagsync_password=Rangerpwd@123
> rangerUsersync_password=Rangerpwd@123
> keyadmin_password=Rangerpwd@123
> policymgr_external_url=https://hostname_ranger:6182
> policymgr_http_enabled=false
> policymgr_https_keystore_file=/opt/iss_cert/clientcert.jks
> policymgr_https_keystore_keyalias=
> policymgr_https_keystore_password=31b17532aeb4fb5ba3af2bae850567
> unix_user=ranger
> unix_user_pwd=Rangerpwd@123
> unix_group=ranger
> #LDAP|ACTIVE_DIRECTORY|UNIX|NONE
> authentication_method=LDAP
> xa_ldap_url=ldaps://hostname_ldapserver:636
> xa_ldap_userDNpattern=uid=\{0},OU=xxx,DC=xx,DC=,DC=COM
> xa_ldap_groupSearchBase=DC=xxx,DC=ccc,DC=COM
> xa_ldap_groupSearchFilter=(member=cn=\{0},OU=xxx,DC=xx,DC=,DC=COM)
> xa_ldap_groupRoleAttribute=cn
> xa_ldap_base_dn=DC=xx,DC=,DC=COM
> xa_ldap_bind_dn=CN=XXX,OU=XX,DC=xx,DC=,DC=COM
> xa_ldap_bind_password=uBLRzVJK
> xa_ldap_referral=follow
> xa_ldap_userSearchFilter=(uid=\{0})
> *With the above values,able to start ranger with SSL and LDAP enabled and
> also able to login succesfully with both unix admin credentials and also with
> ldap credentials.*
>
> *ranger-2.1.0-SNAPSHOT-presto-plugin/install.properties:-*
> POLICY_MGR_URL=https:/hostname_ranger:6182
> REPOSITORY_NAME=presto-catalogs-dev
> *# You do not need use SSL between agent and security admin tool, please
> leave these sample value as it is.*
> SSL_KEYSTORE_FILE_PATH=/etc/hadoop/conf/ranger-plugin-keystore.jks
> SSL_KEYSTORE_PASSWORD=none
> SSL_TRUSTSTORE_FILE_PATH=/etc/hadoop/conf/ranger-plugin-truststore.jks
> SSL_TRUSTSTORE_PASSWORD=none
> *keep blank if component user is default*
> CUSTOM_USER=
> *keep blank if component group is default*
> CUSTOM_GROUP=
>
> *presto-server-310/etc/config.properties:-*
> coordinator=true
> node-scheduler.include-coordinator=true
> http-server.http.enabled=false
> node.internal-address-source=FQDN
> node.internal-address=hostname_presto
> internal-communication.https.required=true
> internal-communication.https.keystore.path=/opt/iss_cert/clientcert.jks
> internal-communication.https.keystore.key=31b17532aeb4fb5ba3af2bae850567
> discovery-server.enabled=true
> discovery.uri=https://hostname_presto:8443
> http-server.authentication.type=PASSWORD,CERTIFICATE
> http-server.https.enabled=true
> http-server.https.port=8443
> http-server.https.keystore.path=/opt/iss_cert/clientcert.jks
> http-server.https.keystore.key=31b17532aeb4fb5ba3af2bae850567
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (RANGER-2754) Update presto dependency and implement row/column level security
Bolke de Bruin created RANGER-2754: -- Summary: Update presto dependency and implement row/column level security Key: RANGER-2754 URL: https://issues.apache.org/jira/browse/RANGER-2754 Project: Ranger Issue Type: Improvement Components: plugins Affects Versions: master Reporter: Bolke de Bruin 1. PrestoSql has changed its Security API hence the Ranger plugin has stopped working for versions > ~321. 2. Presto master now has row/column level security support -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2502) Presto plugin insert bug
[ https://issues.apache.org/jira/browse/RANGER-2502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16883248#comment-16883248 ] Bolke de Bruin commented on RANGER-2502: [~abhayk] can you please merge the patch? > Presto plugin insert bug > > > Key: RANGER-2502 > URL: https://issues.apache.org/jira/browse/RANGER-2502 > Project: Ranger > Issue Type: Bug > Components: admin, plugins >Affects Versions: 2.0.0 >Reporter: Pedro Gonçalves Rossi Rodrigues >Priority: Major > Attachments: > 0001-RANGER-2502-Correct-service-definition-for-presto.patch > > > I wasn't able to insert anything even with the admin user until I changed > from INSERT to UPDATE on the following line > [https://github.com/apache/ranger/blob/master/plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java#L458] > , I don't know if this is the admin fault because it is not showing the > INSERT permission or it really should use UPDATE instead of INSERT, but I > think it makes more sense to show INSERT instead of UPDATE since presto sql > syntax doesn't have an UPDATE statement > (https://prestosql.io/docs/current/sql.html) -- This message was sent by Atlassian JIRA (v7.6.14#76016)
Re: Review Request 71037: Fix service definition for Presto
> On July 8, 2019, 11 p.m., Abhay Kulkarni wrote: > > agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json > > Line 105 (original), 105 (patched) > > <https://reviews.apache.org/r/71037/diff/1/?file=2154024#file2154024line105> > > > > Please note that a change to Presto service-definition requires that if > > presto service is installed already, then it needs to be updated externally > > (using cURL or Java client), before the changes can be effective. This does > > not apply for new installation or Presto plugin. > > Bolke de Bruin wrote: > this isnt in any release yet so is this "upgrade path" required then? > where can i find where upgrades are executed? > > Velmurugan Periasamy wrote: > Since this is not in a release, it might be ok. > > For reference, service defs are updated during upgrade with a java patch. > See > https://github.com/apache/ranger/blob/master/security-admin/src/main/java/org/apache/ranger/patch/PatchForHiveServiceDefUpdate_J10030.java Will keep that in mind for a potential follow up. Thx. - Bolke --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71037/#review216431 --- On July 8, 2019, 8:12 p.m., Bolke de Bruin wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71037/ > --- > > (Updated July 8, 2019, 8:12 p.m.) > > > Review request for ranger and Abhay Kulkarni. > > > Bugs: https://issues.apache.org/jira/browse/RANGER-2502 > > https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/RANGER-2502 > > > Repository: ranger > > > Description > --- > > Fix service definition for Presto (update was used instead of insert) > > > Diffs > - > > agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json > f0ecf6b43 > > > Diff: https://reviews.apache.org/r/71037/diff/1/ > > > Testing > --- > > Tested locally > > > Thanks, > > Bolke de Bruin > >
Re: Review Request 71037: Fix service definition for Presto
> On jul 8, 2019, 11 p.m., Abhay Kulkarni wrote: > > agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json > > Line 105 (original), 105 (patched) > > <https://reviews.apache.org/r/71037/diff/1/?file=2154024#file2154024line105> > > > > Please note that a change to Presto service-definition requires that if > > presto service is installed already, then it needs to be updated externally > > (using cURL or Java client), before the changes can be effective. This does > > not apply for new installation or Presto plugin. this isnt in any release yet so is this "upgrade path" required then? where can i find where upgrades are executed? - Bolke --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71037/#review216431 ----------- On jul 8, 2019, 8:12 p.m., Bolke de Bruin wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71037/ > --- > > (Updated jul 8, 2019, 8:12 p.m.) > > > Review request for ranger and Abhay Kulkarni. > > > Bugs: https://issues.apache.org/jira/browse/RANGER-2502 > > https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/RANGER-2502 > > > Repository: ranger > > > Description > --- > > Fix service definition for Presto (update was used instead of insert) > > > Diffs > - > > agents-common/src/main/resources/service-defs/ranger-servicedef-presto.json > f0ecf6b43 > > > Diff: https://reviews.apache.org/r/71037/diff/1/ > > > Testing > --- > > Tested locally > > > Thanks, > > Bolke de Bruin > >
