Review Request 73480: RANGER-3545 : Default Ranger policy for KMS should include "om" user for Ozone bucket level encryption to work.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73480/ --- Review request for ranger, Dhaval Shah, Abhay Kulkarni, Mehul Parikh, Pradeep Agrawal, and Velmurugan Periasamy. Bugs: RANGER-3545 https://issues.apache.org/jira/browse/RANGER-3545 Repository: ranger Description --- Default Ranger policy for KMS should include "om" user for Ozone bucket level encryption to work. Diffs - plugin-kms/src/main/java/org/apache/ranger/services/kms/RangerServiceKMS.java 8af592b2a security-admin/src/main/resources/conf.dist/ranger-admin-site.xml 12eb8fe64 Diff: https://reviews.apache.org/r/73480/diff/1/ Testing --- Ran : mvn clean compile test verify install - Build was successful Testing : Performed the functional testing. Thanks, Mateen Mansoori
[GitHub] [ranger] takezoe commented on pull request #85: RANGER-3221: Improve logging in Presto plugin
takezoe commented on pull request #85: URL: https://github.com/apache/ranger/pull/85#issuecomment-885986928 Reviewed on ReviewBoard: https://reviews.apache.org/r/73424/ and merged: https://github.com/apache/ranger/commit/f0f1877c08a2c92d6de537121106da7decdd5343 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [ranger] takezoe closed pull request #85: RANGER-3221: Improve logging in Presto plugin
takezoe closed pull request #85: URL: https://github.com/apache/ranger/pull/85 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [ranger] alvaroqueiroz commented on pull request #110: Python ranger_client call_api - add case for 404 response
alvaroqueiroz commented on pull request #110: URL: https://github.com/apache/ranger/pull/110#issuecomment-885939343 @mneethiraj thank you for your comments, the changes make sense. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [ranger] alvaroqueiroz commented on a change in pull request #110: Python ranger_client call_api - add case for 404 response
alvaroqueiroz commented on a change in pull request #110:
URL: https://github.com/apache/ranger/pull/110#discussion_r675888346
##
File path: intg/src/main/python/apache_ranger/client/ranger_client.py
##
@@ -324,6 +324,11 @@ def __call_api(self, api, query_params=None,
request_data=None):
LOG.error("Ranger admin unavailable. HTTP Status: %s",
HTTPStatus.SERVICE_UNAVAILABLE)
ret = None
+elif response.status_code == HTTPStatus.SERVICE_NOT_FOUND:
Review comment:
thanks for the comments @mneethiraj, it makes sense
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [ranger] mneethiraj commented on a change in pull request #110: Python ranger_client call_api - add case for 404 response
mneethiraj commented on a change in pull request #110:
URL: https://github.com/apache/ranger/pull/110#discussion_r675874781
##
File path: intg/src/main/python/apache_ranger/client/ranger_client.py
##
@@ -324,6 +324,11 @@ def __call_api(self, api, query_params=None,
request_data=None):
LOG.error("Ranger admin unavailable. HTTP Status: %s",
HTTPStatus.SERVICE_UNAVAILABLE)
ret = None
+elif response.status_code == HTTPStatus.SERVICE_NOT_FOUND:
+LOG.error("Ranger service not found. HTTP Status: %s",
HTTPStatus.SERVICE_NOT_FOUND)
Review comment:
"Ranger service not found" => "Not found"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [ranger] mneethiraj commented on a change in pull request #110: Python ranger_client call_api - add case for 404 response
mneethiraj commented on a change in pull request #110:
URL: https://github.com/apache/ranger/pull/110#discussion_r675874536
##
File path: intg/src/main/python/apache_ranger/client/ranger_client.py
##
@@ -324,6 +324,11 @@ def __call_api(self, api, query_params=None,
request_data=None):
LOG.error("Ranger admin unavailable. HTTP Status: %s",
HTTPStatus.SERVICE_UNAVAILABLE)
ret = None
+elif response.status_code == HTTPStatus.SERVICE_NOT_FOUND:
Review comment:
Consider renaming SERVICE_NOT_FOUND => NOT_FOUND.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: Review Request 73479: RANGER-3342 : Addendum fix Need to make the Ranger embedded server work directory configurable
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73479/#review223263 --- Ship it! Ship It! - Madhan Neethiraj On July 23, 2021, 7:32 p.m., Vishal Suvagia wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73479/ > --- > > (Updated July 23, 2021, 7:32 p.m.) > > > Review request for ranger, Ankita Sinha, Dhaval Shah, Dineshkumar Yadav, > Gautam Borad, Jayendra Parab, Kishor Gollapalliwar, Abhay Kulkarni, Madhan > Neethiraj, Mehul Parikh, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, > and Velmurugan Periasamy. > > > Bugs: RANGER-3342 > https://issues.apache.org/jira/browse/RANGER-3342 > > > Repository: ranger > > > Description > --- > > Earlier fix included a Logger check for Log Level Fine, but the LOG get level > is found to be null leading to a > NullPointerException. > > > Diffs > - > > > embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/EmbeddedServer.java > d3b10845556a20915f37e84b26ad050791e5495e > > > Diff: https://reviews.apache.org/r/73479/diff/1/ > > > Testing > --- > > Validated on a cluster. > > > Thanks, > > Vishal Suvagia > >
Review Request 73479: RANGER-3342 : Addendum fix Need to make the Ranger embedded server work directory configurable
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73479/ --- Review request for ranger, Ankita Sinha, Dhaval Shah, Dineshkumar Yadav, Gautam Borad, Jayendra Parab, Kishor Gollapalliwar, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy. Bugs: RANGER-3342 https://issues.apache.org/jira/browse/RANGER-3342 Repository: ranger Description --- Earlier fix included a Logger check for Log Level Fine, but the LOG get level is found to be null leading to a NullPointerException. Diffs - embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/EmbeddedServer.java d3b10845556a20915f37e84b26ad050791e5495e Diff: https://reviews.apache.org/r/73479/diff/1/ Testing --- Validated on a cluster. Thanks, Vishal Suvagia
[jira] [Updated] (RANGER-3342) Need to make the Ranger embedded server work directory configurable
[ https://issues.apache.org/jira/browse/RANGER-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vishal Suvagia updated RANGER-3342: --- Attachment: RANGER-3342.02.patch > Need to make the Ranger embedded server work directory configurable > --- > > Key: RANGER-3342 > URL: https://issues.apache.org/jira/browse/RANGER-3342 > Project: Ranger > Issue Type: Improvement > Components: admin >Affects Versions: 2.1.0, 3.0.0 >Reporter: Vishal Suvagia >Assignee: Vishal Suvagia >Priority: Major > Attachments: RANGER-3342.01.patch, RANGER-3342.02.patch, > RANGER-3342.patch > > > Currently the work directory for Ranger embedded server is not configurable. > Need to make the work directory configurable to a custom location so that > user can customize if required. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (RANGER-3347) Add default policy for hbase user in hdfs services
Kishor Gollapalliwar created RANGER-3347: Summary: Add default policy for hbase user in hdfs services Key: RANGER-3347 URL: https://issues.apache.org/jira/browse/RANGER-3347 Project: Ranger Issue Type: Improvement Components: Ranger Reporter: Kishor Gollapalliwar Assignee: Kishor Gollapalliwar Add default policy for hbase user in hdfs services as follows. *Details:* * User = hbase * Permissions = read + write + execute * path = /hbase/archive -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 73452: RANGER-3023: Permission tab takes longer time to load with large number of users and group_users data
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73452/#review223261 --- As an extension, since only username & groupname are used from XXUser & XXGroup objects, it is possible to add sql query to return map of id & name instead of the actual objects? - Sailaja Polavarapu On July 20, 2021, 1:52 p.m., Mahesh Bandal wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73452/ > --- > > (Updated July 20, 2021, 1:52 p.m.) > > > Review request for ranger, Ankita Sinha, Dhaval Shah, Dineshkumar Yadav, > Gautam Borad, Kishor Gollapalliwar, Abhay Kulkarni, Madhan Neethiraj, Mehul > Parikh, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, and Velmurugan > Periasamy. > > > Bugs: RANGER-3023 > https://issues.apache.org/jira/browse/RANGER-3023 > > > Repository: ranger > > > Description > --- > > GET API /service/xusers/permission takes longer time to load with following > number of users and group mappings in db. > > select count(*) from x_user; > 109040 > > select count(*) from x_portal_user; > 109038 > > select count(*) from x_group_users; > 689952 > > Current problem : For every ModuleDef, db call to fetch all XXUser, > XXPortalUser and creating a Map object using > xUserService.getXXPortalUserIdXXUserMap() is a costly operation. Similarly > for xGroupService.getXXGroupIdXXGroupMap(). > > Solution: > In the following patch, I have overriedden searchModuleDef function in > XModuleDefService which will fetch users and groups only once. > i.e. Map xXPortalUserIdXXUserMap = > xUserService.getXXPortalUserIdXXUserMap(); > Map xXGroupMap = xGroupService.getXXGroupIdXXGroupMap(); > > These two objects will be passed to an overloaded method populateViewBean() > > > Diffs > - > > > security-admin/src/main/java/org/apache/ranger/service/XModuleDefService.java > d5ca38548 > > > Diff: https://reviews.apache.org/r/73452/diff/2/ > > > Testing > --- > > xUserService.getXXPortalUserIdXXUserMap() takes approximately 2000 > milliseconds. > xGroupService.getXXGroupIdXXGroupMap() takes approximately 500 milliseconds. > > Before patch, XModuleDefServiceBase.searchModuleDef() took 30252 milliseconds. > After patch, XModuleDefService.searchModuleDef() took 13766 milliseconds. > > GET API /service/xusers/permission response improved by ~16 seconds for the > above mentioned dataset. > > > Thanks, > > Mahesh Bandal > >
[jira] [Resolved] (RANGER-2348) Error getting policies for hiveserver2 Interactive
[
https://issues.apache.org/jira/browse/RANGER-2348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bhavik Patel resolved RANGER-2348.
--
Resolution: Not A Problem
> Error getting policies for hiveserver2 Interactive
> --
>
> Key: RANGER-2348
> URL: https://issues.apache.org/jira/browse/RANGER-2348
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Affects Versions: 1.2.0
>Reporter: Davide Vergari
>Priority: Major
>
> Ranger Admin REST Client fails on calling getServicePoliciesIfUpdated method
> if running inside hiveserver2 Interactive.
> The stack trace is the following:
>
> {code:java}
> ERROR [Thread-9]: client.RangerAdminRESTClient
> (RangerAdminRESTClient.java:getServicePoliciesIfUpdated(167)) - Error getting
> policies for serviceName=ValuePartner_hiveresponse=null
> com.sun.jersey.spi.inject.Errors$ErrorMessagesException: null
> at com.sun.jersey.spi.inject.Errors.processErrorMessages(Errors.java:170)
> ~[jersey-core-1.19.3.jar:1.19.3]
> at com.sun.jersey.spi.inject.Errors.postProcess(Errors.java:136)
> ~[jersey-core-1.19.3.jar:1.19.3]
> at com.sun.jersey.spi.inject.Errors.processWithErrors(Errors.java:199)
> ~[jersey-core-1.19.3.jar:1.19.3]
> at com.sun.jersey.api.client.Client.(Client.java:188)
> ~[jersey-client-1.19.jar:1.19]
> at com.sun.jersey.api.client.Client.(Client.java:171)
> ~[jersey-client-1.19.jar:1.19]
> at com.sun.jersey.api.client.Client.create(Client.java:683)
> ~[jersey-client-1.19.jar:1.19]
> at
> org.apache.ranger.plugin.util.RangerRESTClient.buildClient(RangerRESTClient.java:211)
> ~[ranger-plugins-common-1.2.0.3.1.0.0-78.jar:1.2.0.3.1.0.0-78]
> at
> org.apache.ranger.plugin.util.RangerRESTClient.getClient(RangerRESTClient.java:176)
> ~[ranger-plugins-common-1.2.0.3.1.0.0-78.jar:1.2.0.3.1.0.0-78]
> at
> org.apache.ranger.plugin.util.RangerRESTClient.getResource(RangerRESTClient.java:156)
> ~[ranger-plugins-common-1.2.0.3.1.0.0-78.jar:1.2.0.3.1.0.0-78]
> at
> org.apache.ranger.admin.client.RangerAdminRESTClient.createWebResource(RangerAdminRESTClient.java:279)
> ~[ranger-plugins-common-1.2.0.3.1.0.0-78.jar:1.2.0.3.1.0.0-78]
> at
> org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:128)
> [ranger-plugins-common-1.2.0.3.1.0.0-78.jar:1.2.0.3.1.0.0-78]
> at
> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:264)
> [ranger-plugins-common-1.2.0.3.1.0.0-78.jar:1.2.0.3.1.0.0-78]
> at
> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:202)
> [ranger-plugins-common-1.2.0.3.1.0.0-78.jar:1.2.0.3.1.0.0-78]
> at
> org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:171)
> [ranger-plugins-common-1.2.0.3.1.0.0-78.jar:1.2.0.3.1.0.0-78]
> {code}
> The error seems generated by the buildClient() method that always returns
> null. Running the plugin inside hiveserver2 (non interactive) works well.
>
> Current workaround to use updated policies from ranger admin is to install
> both hiveserver2 and hiveserver2 Interactive on the same host
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (RANGER-2128) Implement SparkSQL plugin
[ https://issues.apache.org/jira/browse/RANGER-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386314#comment-17386314 ] Bhavik Patel commented on RANGER-2128: -- [~Qin Yao] [~rmartine] [~toopt4] can this Jira be merged? > Implement SparkSQL plugin > - > > Key: RANGER-2128 > URL: https://issues.apache.org/jira/browse/RANGER-2128 > Project: Ranger > Issue Type: New Feature > Components: plugins, Ranger >Affects Versions: 1.1.0 >Reporter: t oo >Assignee: Kent Yao >Priority: Major > Attachments: support_ranger11.tgz > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Implement SparkSQL plugin -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 73452: RANGER-3023: Permission tab takes longer time to load with large number of users and group_users data
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73452/#review223258 --- Ship it! Ship It! - Pradeep Agrawal On July 20, 2021, 1:52 p.m., Mahesh Bandal wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73452/ > --- > > (Updated July 20, 2021, 1:52 p.m.) > > > Review request for ranger, Ankita Sinha, Dhaval Shah, Dineshkumar Yadav, > Gautam Borad, Kishor Gollapalliwar, Abhay Kulkarni, Madhan Neethiraj, Mehul > Parikh, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, and Velmurugan > Periasamy. > > > Bugs: RANGER-3023 > https://issues.apache.org/jira/browse/RANGER-3023 > > > Repository: ranger > > > Description > --- > > GET API /service/xusers/permission takes longer time to load with following > number of users and group mappings in db. > > select count(*) from x_user; > 109040 > > select count(*) from x_portal_user; > 109038 > > select count(*) from x_group_users; > 689952 > > Current problem : For every ModuleDef, db call to fetch all XXUser, > XXPortalUser and creating a Map object using > xUserService.getXXPortalUserIdXXUserMap() is a costly operation. Similarly > for xGroupService.getXXGroupIdXXGroupMap(). > > Solution: > In the following patch, I have overriedden searchModuleDef function in > XModuleDefService which will fetch users and groups only once. > i.e. Map xXPortalUserIdXXUserMap = > xUserService.getXXPortalUserIdXXUserMap(); > Map xXGroupMap = xGroupService.getXXGroupIdXXGroupMap(); > > These two objects will be passed to an overloaded method populateViewBean() > > > Diffs > - > > > security-admin/src/main/java/org/apache/ranger/service/XModuleDefService.java > d5ca38548 > > > Diff: https://reviews.apache.org/r/73452/diff/2/ > > > Testing > --- > > xUserService.getXXPortalUserIdXXUserMap() takes approximately 2000 > milliseconds. > xGroupService.getXXGroupIdXXGroupMap() takes approximately 500 milliseconds. > > Before patch, XModuleDefServiceBase.searchModuleDef() took 30252 milliseconds. > After patch, XModuleDefService.searchModuleDef() took 13766 milliseconds. > > GET API /service/xusers/permission response improved by ~16 seconds for the > above mentioned dataset. > > > Thanks, > > Mahesh Bandal > >
Review Request 73474: RANGER-3346 In case of Solr Ranger plugin use custom HttpClient in SolrClient
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73474/ --- Review request for ranger and Ramesh Mani. Repository: ranger Description --- I made the SolrAuditDestination class to be aware if it's initialized by the Ranger Solr plugin and in this case prevents the usage of HttpClientUtil from SolrJ lib but setup it by its own. The way how the SolrClient and its HttpClient build is identical to how it's implemented in the HttpClientUtil class (except the interceptor which causes troubles). Diffs - agents-audit/src/main/java/org/apache/ranger/audit/destination/SolrAuditDestination.java 8532bf8e9 agents-audit/src/main/java/org/apache/ranger/audit/provider/AuditProviderFactory.java 7a3c7f61d agents-audit/src/main/java/org/apache/ranger/audit/provider/solr/SolrAuditProvider.java 1ef115a87 agents-audit/src/main/java/org/apache/ranger/audit/utils/SolrAppUtil.java 5cb8b1be1 agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java 82b048194 Diff: https://reviews.apache.org/r/73474/diff/1/ Testing --- Thanks, Geza Nagy
[jira] [Created] (RANGER-3346) Ranger Solr Plugin sends audits with PKI auth instead of Kerberos
Geza Nagy created RANGER-3346: - Summary: Ranger Solr Plugin sends audits with PKI auth instead of Kerberos Key: RANGER-3346 URL: https://issues.apache.org/jira/browse/RANGER-3346 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Geza Nagy SolrAuditDestination class connects to Solr by building client with HttpClientUtil (from SolrJ library). It's fine except in case of Solr Ranger Plugin. The Solr server uses the SolrJ lib as well for communication among its nodes and sets static/global configuration inside the HttpClientUtil which makes the SolrAuditDestination class's client to misbehave as sending the audits request with a header which forces the authentication to PKI. Prior to Solr 8.8 it leaded to only warning level Exceptions but after this version this will cause failure when sending audits. Relevant Solr JIRA about changing the implementation of HttpClientUtil class: https://issues.apache.org/jira/browse/SOLR-15388 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2174) Write unit test for Ranger Elasticsearch plugin RANGER-2170
[ https://issues.apache.org/jira/browse/RANGER-2174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386192#comment-17386192 ] Bhavik Patel commented on RANGER-2174: -- [~zhangqiang2] Are you working on this Jira? > Write unit test for Ranger Elasticsearch plugin RANGER-2170 > --- > > Key: RANGER-2174 > URL: https://issues.apache.org/jira/browse/RANGER-2174 > Project: Ranger > Issue Type: Sub-task > Components: Ranger >Affects Versions: master >Reporter: Qiang Zhang >Assignee: Qiang Zhang >Priority: Major > Labels: new-feature, patch, unit-test > > Write unit test for Ranger Elasticsearch plugin RANGER-2170 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (RANGER-1202) flume Elasticsearch 5.0 support
[ https://issues.apache.org/jira/browse/RANGER-1202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavik Patel resolved RANGER-1202. -- Resolution: Invalid > flume Elasticsearch 5.0 support > --- > > Key: RANGER-1202 > URL: https://issues.apache.org/jira/browse/RANGER-1202 > Project: Ranger > Issue Type: Bug > Components: admin >Reporter: tycho_yang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2926) Issue in setting up Audit Log with ElasticSearch
[
https://issues.apache.org/jira/browse/RANGER-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386191#comment-17386191
]
Bhavik Patel commented on RANGER-2926:
--
[~bdasari] I think you have to update your ES version to *7.6.0*.
If your issue got resolved then can you please close this Jira?
> Issue in setting up Audit Log with ElasticSearch
> -
>
> Key: RANGER-2926
> URL: https://issues.apache.org/jira/browse/RANGER-2926
> Project: Ranger
> Issue Type: Bug
> Components: Ranger
>Affects Versions: 2.0.0
>Reporter: Bhanu
>Priority: Major
>
> Hi,
> We are using Ranger 2.1.0.
> Trying to setup AuditLog with ElasticSearch Server having version 7.0.1
> We have configured the Ranger with all details but there is an error that is
> keep on coming as below. Please let me know where we are going wrong here. We
> have tried recreating the index multiple times with all below parameters
> 2020-07-27T13:08:35.233Z ERROR org.apache.ranger.audit.queue.AuditBatchQueue0
> org.apache.ranger.audit.provider.BaseAuditHandler Error sending message to
> ElasticSearch
> org.elasticsearch.action.ActionRequestValidationException: Validation Failed:
> 1: type is missing;2: type is missing;
> at org.elasticsearch.action.bulk.BulkRequest.validate(BulkRequest.java:393)
> at
> org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1480)
> at
> org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1454)
> at
> org.elasticsearch.client.RestHighLevelClient.bulk(RestHighLevelClient.java:497)
> at
> org.apache.ranger.audit.destination.ElasticSearchAuditDestination.log(ElasticSearchAuditDestination.java:125)
> at
> org.apache.ranger.audit.queue.AuditBatchQueue.runLogAudit(AuditBatchQueue.java:309)
> at
> org.apache.ranger.audit.queue.AuditBatchQueue.run(AuditBatchQueue.java:215)
> at java.base/java.lang.Thread.run(Thread.java:834)
> 2020-07-27T13:08:35.233Z WARN org.apache.ranger.audit.queue.AuditBatchQueue0
> org.apache.ranger.audit.provider.BaseAuditHandler failed to log audit event:
> \{"repoType":17,"repo":"prestostg-tkg","reqUser":"bdasari","evtTime":"2020-07-27
>
> 13:08:35.102","resource":"hive_stg/ref_maritz","resType":"schema","action":"select","result":1,"agent":"presto","policy":21,"enforcer":"ranger-acl","agentHost":"coordinator2-694c5dbbb6-msh58","logType":"RangerAudit","id":"f733c835-c9ee-4507-b917-9eb822303d2b-792211","seq_num":1584423,"event_count":1,"event_dur_ms":0,"tags":[],"cluster_name":"","policy_version":148},
> errorMessage=
> 2020-07-27T13:08:35.233Z WARN org.apache.ranger.audit.queue.AuditBatchQueue0
> org.apache.ranger.audit.provider.BaseAuditHandler Log failure count: 4 in
> past 01:30.003 minutes; 792212 during process lifetime
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (RANGER-3326) get db audit log failed via rest api
[
https://issues.apache.org/jira/browse/RANGER-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bhavik Patel resolved RANGER-3326.
--
Resolution: Invalid
> get db audit log failed via rest api
>
>
> Key: RANGER-3326
> URL: https://issues.apache.org/jira/browse/RANGER-3326
> Project: Ranger
> Issue Type: Bug
> Components: audit
>Affects Versions: 2.1.0
>Reporter: Xuze Yang
>Priority: Major
>
> Using ranger2.1.0, we store audit logs in db(although log in db is deprecated
> since ranger0.6, we still use it because we don't want to introduce and
> maintain a new component like solr or elasticSearch). The audit log
> information can be successfully stored in db. But when we want to get audit
> logs via REST api(/service/assets/accessAudit), it throws following
> exceptions:
> {code:java}
> Jul 01, 2021 5:09:59 PM com.sun.jersey.spi.container.ContainerResponse
> mapMappableContainerExceptionJul 01, 2021 5:09:59 PM
> com.sun.jersey.spi.container.ContainerResponse
> mapMappableContainerExceptionSEVERE: The RuntimeException could not be mapped
> to a response, re-throwing to the HTTP containerjava.lang.ClassCastException:
> java.util.ArrayList cannot be cast to java.lang.String at
> org.apache.ranger.common.SearchUtil.buildWhereClause(SearchUtil.java:518) at
> org.apache.ranger.common.SearchUtil.buildWhereClause(SearchUtil.java:384) at
> org.apache.ranger.common.SearchUtil.createSearchQuery(SearchUtil.java:708) at
> org.apache.ranger.service.AbstractBaseResourceService.createQuery(AbstractBaseResourceService.java:519)
> at
> org.apache.ranger.service.AbstractBaseResourceService.getCountForSearchQuery(AbstractBaseResourceService.java:535)
> at
> org.apache.ranger.service.AbstractBaseResourceService.searchResources(AbstractBaseResourceService.java:564)
> at
> org.apache.ranger.service.XAccessAuditService.searchXAccessAudits(XAccessAuditService.java:172)
> at org.apache.ranger.biz.AssetMgr.getAccessLogs(AssetMgr.java:1132) at
> org.apache.ranger.rest.AssetREST.getAccessLogs(AssetREST.java:671) at
> org.apache.ranger.rest.AssetREST$$FastClassBySpringCGLIB$$8cffcb6d.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:737)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:69)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
> at
> org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:283)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:672)
> at
> org.apache.ranger.rest.AssetREST$$EnhancerBySpringCGLIB$$9c5e18ed.getAccessLogs()
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
> at
>
[jira] [Commented] (RANGER-3326) get db audit log failed via rest api
[
https://issues.apache.org/jira/browse/RANGER-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386187#comment-17386187
]
Bhavik Patel commented on RANGER-3326:
--
[~Xuze Yang] Audit to DB support is not maintained(removed), you can use Solr,
ES, or Hdfs.
> get db audit log failed via rest api
>
>
> Key: RANGER-3326
> URL: https://issues.apache.org/jira/browse/RANGER-3326
> Project: Ranger
> Issue Type: Bug
> Components: audit
>Affects Versions: 2.1.0
>Reporter: Xuze Yang
>Priority: Major
>
> Using ranger2.1.0, we store audit logs in db(although log in db is deprecated
> since ranger0.6, we still use it because we don't want to introduce and
> maintain a new component like solr or elasticSearch). The audit log
> information can be successfully stored in db. But when we want to get audit
> logs via REST api(/service/assets/accessAudit), it throws following
> exceptions:
> {code:java}
> Jul 01, 2021 5:09:59 PM com.sun.jersey.spi.container.ContainerResponse
> mapMappableContainerExceptionJul 01, 2021 5:09:59 PM
> com.sun.jersey.spi.container.ContainerResponse
> mapMappableContainerExceptionSEVERE: The RuntimeException could not be mapped
> to a response, re-throwing to the HTTP containerjava.lang.ClassCastException:
> java.util.ArrayList cannot be cast to java.lang.String at
> org.apache.ranger.common.SearchUtil.buildWhereClause(SearchUtil.java:518) at
> org.apache.ranger.common.SearchUtil.buildWhereClause(SearchUtil.java:384) at
> org.apache.ranger.common.SearchUtil.createSearchQuery(SearchUtil.java:708) at
> org.apache.ranger.service.AbstractBaseResourceService.createQuery(AbstractBaseResourceService.java:519)
> at
> org.apache.ranger.service.AbstractBaseResourceService.getCountForSearchQuery(AbstractBaseResourceService.java:535)
> at
> org.apache.ranger.service.AbstractBaseResourceService.searchResources(AbstractBaseResourceService.java:564)
> at
> org.apache.ranger.service.XAccessAuditService.searchXAccessAudits(XAccessAuditService.java:172)
> at org.apache.ranger.biz.AssetMgr.getAccessLogs(AssetMgr.java:1132) at
> org.apache.ranger.rest.AssetREST.getAccessLogs(AssetREST.java:671) at
> org.apache.ranger.rest.AssetREST$$FastClassBySpringCGLIB$$8cffcb6d.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:737)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:69)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
> at
> org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:283)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:672)
> at
> org.apache.ranger.rest.AssetREST$$EnhancerBySpringCGLIB$$9c5e18ed.getAccessLogs()
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at
>
[jira] [Commented] (RANGER-3308) Create python script to test stability of policy CRUD
[ https://issues.apache.org/jira/browse/RANGER-3308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386178#comment-17386178 ] Bhavik Patel commented on RANGER-3308: -- Thanks [~kishor.gollapalliwar] this is very handy for the dev testing. Here I think we can avoid bash script if we have only a python script then maintenance will be easy. [~rmani] / [~madhan] / [~pradeepagrawal8184] any thoughts on this. > Create python script to test stability of policy CRUD > - > > Key: RANGER-3308 > URL: https://issues.apache.org/jira/browse/RANGER-3308 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Affects Versions: 3.0.0 >Reporter: Kishor Gollapalliwar >Assignee: Kishor Gollapalliwar >Priority: Major > > Create a python script which will create & delete policies multiple times > based on input. This script will be more helpful testing stability of Ranger > Admin policy creation framework. The bash script will execute python script > concurrently (default=5), each execution will run for few cycles(default=10), > each cycle will create/get, update/get, delete/get policy items for given > ranger service. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (RANGER-3308) Create python script to test stability of policy CRUD
[ https://issues.apache.org/jira/browse/RANGER-3308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kishor Gollapalliwar updated RANGER-3308: - Description: Create a python script which will create & delete policies multiple times based on input. This script will be more helpful testing stability of Ranger Admin policy creation framework. The bash script will execute python script concurrently (default=5), each execution will run for few cycles(default=10), each cycle will create/get, update/get, delete/get policy items for given ranger service.(was: Create a python script which will create & delete policies multiple times based on input) > Create python script to test stability of policy CRUD > - > > Key: RANGER-3308 > URL: https://issues.apache.org/jira/browse/RANGER-3308 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Affects Versions: 3.0.0 >Reporter: Kishor Gollapalliwar >Assignee: Kishor Gollapalliwar >Priority: Major > > Create a python script which will create & delete policies multiple times > based on input. This script will be more helpful testing stability of Ranger > Admin policy creation framework. The bash script will execute python script > concurrently (default=5), each execution will run for few cycles(default=10), > each cycle will create/get, update/get, delete/get policy items for given > ranger service. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [ranger] chia7712 commented on pull request #110: Python ranger_client call_api - add case for 404 response
chia7712 commented on pull request #110: URL: https://github.com/apache/ranger/pull/110#issuecomment-885459813 > I mean, is there any active ranger committer i can call? Sorry that I'm newbie in ranger community :( -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
