[jira] [Updated] (RANGER-3457) [Session Timeout-Ranger]With multiple tabs if one tab encounters session idle timeout other active tab still continues with old/invalid session cookie.
[ https://issues.apache.org/jira/browse/RANGER-3457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nitin Galave updated RANGER-3457: - Attachment: 0001-RANGER-3457.patch > [Session Timeout-Ranger]With multiple tabs if one tab encounters session idle > timeout other active tab still continues with old/invalid session cookie. > --- > > Key: RANGER-3457 > URL: https://issues.apache.org/jira/browse/RANGER-3457 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Nitin Galave >Assignee: Nitin Galave >Priority: Major > Attachments: 0001-RANGER-3457.patch > > > *Steps:* > 1. Configured ranger.service.inactivity.timeout to 45 sec > 2. Opened ranger UI on multiple tabs. > 3. Left the session idle for 40 secs > 4. once the popup for session idle going to expire shows up clicked on the > "stay logged in" button and performed an operation in one of the tabs. > *Observation* > # Other tabs which were opened removed the RangerSessionID, but the tab > which became active before timeout still using the same RangerSessionID. > # Clicking a link from the above active tab to a new tab still uses the same > RangerSessionID which was removed earlier > # But when clicking ranger ui from CP it opens with a new RangerSessionID > *Note:* > Though using RangerSessionID which was removed in other tabs, i was able to > navigate and perform policy updates. But not sure if any other action will > fail based on session which was removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (RANGER-3457) [Session Timeout-Ranger]With multiple tabs if one tab encounters session idle timeout other active tab still continues with old/invalid session cookie.
Nitin Galave created RANGER-3457: Summary: [Session Timeout-Ranger]With multiple tabs if one tab encounters session idle timeout other active tab still continues with old/invalid session cookie. Key: RANGER-3457 URL: https://issues.apache.org/jira/browse/RANGER-3457 Project: Ranger Issue Type: Improvement Components: Ranger Reporter: Nitin Galave Assignee: Nitin Galave *Steps:* 1. Configured ranger.service.inactivity.timeout to 45 sec 2. Opened ranger UI on multiple tabs. 3. Left the session idle for 40 secs 4. once the popup for session idle going to expire shows up clicked on the "stay logged in" button and performed an operation in one of the tabs. *Observation* # Other tabs which were opened removed the RangerSessionID, but the tab which became active before timeout still using the same RangerSessionID. # Clicking a link from the above active tab to a new tab still uses the same RangerSessionID which was removed earlier # But when clicking ranger ui from CP it opens with a new RangerSessionID *Note:* Though using RangerSessionID which was removed in other tabs, i was able to navigate and perform policy updates. But not sure if any other action will fail based on session which was removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (RANGER-3441) PropertiesUtil (Admin) logging potentially sensitive data
[ https://issues.apache.org/jira/browse/RANGER-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Mani updated RANGER-3441: Fix Version/s: 2.2.0 3.0.0 > PropertiesUtil (Admin) logging potentially sensitive data > - > > Key: RANGER-3441 > URL: https://issues.apache.org/jira/browse/RANGER-3441 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0, 3.0.0 >Reporter: Abhishek Kumar >Assignee: Abhishek Kumar >Priority: Minor > Fix For: 3.0.0, 2.2.0 > > > PropertiesUtil.java line 325 in security-admin logs the key/value pairs which > might include confidential data like passwords. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (RANGER-3441) PropertiesUtil (Admin) logging potentially sensitive data
[ https://issues.apache.org/jira/browse/RANGER-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Mani updated RANGER-3441: Affects Version/s: 3.0.0 2.0.0 > PropertiesUtil (Admin) logging potentially sensitive data > - > > Key: RANGER-3441 > URL: https://issues.apache.org/jira/browse/RANGER-3441 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0, 3.0.0 >Reporter: Abhishek Kumar >Assignee: Abhishek Kumar >Priority: Minor > > PropertiesUtil.java line 325 in security-admin logs the key/value pairs which > might include confidential data like passwords. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (RANGER-3441) PropertiesUtil (Admin) logging potentially sensitive data
[ https://issues.apache.org/jira/browse/RANGER-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhay Kulkarni resolved RANGER-3441. Resolution: Fixed Commit details: master: [https://github.com/apache/ranger/commit/f599c916d84461847613560f856be47438bda884] ranger-2.2: [https://github.com/apache/ranger/commit/f47acb52681c0d8378b15637299f8baf51d0d226] > PropertiesUtil (Admin) logging potentially sensitive data > - > > Key: RANGER-3441 > URL: https://issues.apache.org/jira/browse/RANGER-3441 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Abhishek Kumar >Assignee: Abhishek Kumar >Priority: Minor > > PropertiesUtil.java line 325 in security-admin logs the key/value pairs which > might include confidential data like passwords. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [ranger] ThoSap commented on pull request #115: Add Trino v360 support
ThoSap commented on pull request #115: URL: https://github.com/apache/ranger/pull/115#issuecomment-929645077 Hi, will this be part of the upcoming 2.2.0 release? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: Review Request 73611: RANGER-3449:Updated copyright years in the NOTICE file and plugin details in README.txt
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73611/#review223544 --- Ship it! Ship It! - Velmurugan Periasamy On Sept. 27, 2021, 6:40 a.m., Ramesh Mani wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73611/ > --- > > (Updated Sept. 27, 2021, 6:40 a.m.) > > > Review request for ranger, Don Bosco Durai, Abhay Kulkarni, Madhan Neethiraj, > Mehul Parikh, Pradeep Agrawal, Selvamohan Neethiraj, Sailaja Polavarapu, and > Velmurugan Periasamy. > > > Bugs: RANGER-3449 > https://issues.apache.org/jira/browse/RANGER-3449 > > > Repository: ranger > > > Description > --- > > RANGER-3449:Updated copyright years in the NOTICE file and plugin details in > README.txt > > > Diffs > - > > NOTICE.txt b315087c9 > README.txt fce972ab1 > docs/NOTICE 24e4e687f > > > Diff: https://reviews.apache.org/r/73611/diff/1/ > > > Testing > --- > > - This is for the Apache Release 2.2.0 release effort sub task. > > > Thanks, > > Ramesh Mani > >
Re: Review Request 73615: RANGER-3454:Hive command RELOAD FUNCTION failing in Ranger HiveAuthorization.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73615/#review223543 --- Ship it! Ship It! - Velmurugan Periasamy On Sept. 28, 2021, 1:06 a.m., Ramesh Mani wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73615/ > --- > > (Updated Sept. 28, 2021, 1:06 a.m.) > > > Review request for ranger, Don Bosco Durai, Abhay Kulkarni, Madhan Neethiraj, > Mehul Parikh, Pradeep Agrawal, Selvamohan Neethiraj, Sailaja Polavarapu, and > Velmurugan Periasamy. > > > Bugs: RANGER-3454 > https://issues.apache.org/jira/browse/RANGER-3454 > > > Repository: ranger > > > Description > --- > > RANGER-3454:Hive command RELOAD FUNCTION failing in Ranger HiveAuthorization. > > > Diffs > - > > > hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java > 2ae9379ba > > > Diff: https://reviews.apache.org/r/73615/diff/1/ > > > Testing > --- > > Verified in local VM > > > Thanks, > > Ramesh Mani > >
Re: Review Request 73613: RANGER-3415:Change pom version from 2.2.0-SNAPSHOT to 2.2.0
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73613/#review223542 --- Ship it! Ship It! - Velmurugan Periasamy On Sept. 27, 2021, 8:58 p.m., Ramesh Mani wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73613/ > --- > > (Updated Sept. 27, 2021, 8:58 p.m.) > > > Review request for ranger, Don Bosco Durai, Abhay Kulkarni, Madhan Neethiraj, > Mehul Parikh, Pradeep Agrawal, Selvamohan Neethiraj, Sailaja Polavarapu, and > Velmurugan Periasamy. > > > Bugs: RANGER-3415 > https://issues.apache.org/jira/browse/RANGER-3415 > > > Repository: ranger > > > Description > --- > > RANGER-3415:Change pom version from 2.2.0-SNAPSHOT to 2.2.0 > > > Diffs > - > > agents-audit/pom.xml 485c77f26 > agents-common/pom.xml 9bea6833a > agents-cred/pom.xml 10a5ce71e > agents-installer/pom.xml 9ed3dfc96 > credentialbuilder/pom.xml bb48e0714 > dev-support/ranger-docker/.env 1d45a6229 > distro/pom.xml 314e31d45 > docs/pom.xml 222af4f80 > embeddedwebserver/pom.xml 8d623faf4 > hbase-agent/pom.xml a2ef1a0be > hdfs-agent/pom.xml 5a16993e4 > hive-agent/pom.xml 448971cbb > intg/pom.xml 6b6db5286 > jisql/pom.xml 2d31fdee4 > kms/pom.xml c0cc021dc > knox-agent/pom.xml 1b25603c2 > plugin-atlas/pom.xml 768922b72 > plugin-elasticsearch/pom.xml fa0d005ac > plugin-kafka/pom.xml eae2aab09 > plugin-kms/pom.xml 72e957de7 > plugin-kudu/pom.xml 56c2522ee > plugin-kylin/pom.xml 4b23e351e > plugin-nifi-registry/pom.xml 741f70aca > plugin-nifi/pom.xml 78b0608e3 > plugin-ozone/pom.xml e359b4190 > plugin-presto/pom.xml 56b9aae60 > plugin-schema-registry/pom.xml b457b7713 > plugin-solr/pom.xml 3d07c4e57 > plugin-sqoop/pom.xml 83caa297f > plugin-yarn/pom.xml d008022e3 > pom.xml b1d1d883a > ranger-atlas-plugin-shim/pom.xml 8b74a9ca0 > ranger-elasticsearch-plugin-shim/pom.xml 42bce442b > ranger-examples/conditions-enrichers/pom.xml a9adec2a3 > ranger-examples/distro/pom.xml 342f1d783 > ranger-examples/plugin-sampleapp/pom.xml 2d0e27591 > ranger-examples/pom.xml 3662c5e87 > ranger-examples/sample-client/pom.xml 85f712e48 > ranger-examples/sampleapp/pom.xml a4fbd3b88 > ranger-hbase-plugin-shim/pom.xml 206bb4785 > ranger-hdfs-plugin-shim/pom.xml 09cd4a0a3 > ranger-hive-plugin-shim/pom.xml 4014f5d14 > ranger-kafka-plugin-shim/pom.xml a686da76a > ranger-kms-plugin-shim/pom.xml dced3f334 > ranger-knox-plugin-shim/pom.xml 8aa54dfea > ranger-kylin-plugin-shim/pom.xml ed8fcb7f7 > ranger-ozone-plugin-shim/pom.xml 8c8c27511 > ranger-plugin-classloader/pom.xml 08d9580aa > ranger-presto-plugin-shim/pom.xml e269760f5 > ranger-solr-plugin-shim/pom.xml d8e88cf29 > ranger-sqoop-plugin-shim/pom.xml 3398d8317 > ranger-storm-plugin-shim/pom.xml 8b027ef13 > ranger-tools/pom.xml 676ea9c86 > ranger-util/pom.xml b4eae7429 > ranger-yarn-plugin-shim/pom.xml 67e783a41 > security-admin/pom.xml 76ddd4303 > storm-agent/pom.xml 50ce9b87a > tagsync/pom.xml 2b71a2482 > ugsync-util/pom.xml e8121b6c0 > ugsync/ldapconfigchecktool/ldapconfigcheck/pom.xml 91e9248c8 > ugsync/pom.xml 7f02a9219 > unixauthclient/pom.xml 8fa483768 > unixauthnative/pom.xml 93054a007 > unixauthpam/pom.xml 754835c17 > unixauthservice/pom.xml 2a06dea40 > > > Diff: https://reviews.apache.org/r/73613/diff/1/ > > > Testing > --- > > Verified the build in centos7 > > > Thanks, > > Ramesh Mani > >
[jira] [Created] (RANGER-3456) Improve Kafka Plugin Logging
Joseph Aliase created RANGER-3456:
-
Summary: Improve Kafka Plugin Logging
Key: RANGER-3456
URL: https://issues.apache.org/jira/browse/RANGER-3456
Project: Ranger
Issue Type: Improvement
Components: plugins
Affects Versions: 2.1.0
Reporter: Joseph Aliase
Improve Kafka Plugin:
It would be helpful if we can log a message when a given IP/session is denied
access to a resource. Today entire request is logged in debug mode.
Something like this:
logger.warn("Ip-address ["+ip+"] is not authorized to perform operation
["+operation+"] on resource [" + resource+"]")
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (RANGER-3456) Improve Kafka Plugin Unauthroized Request Logging
[
https://issues.apache.org/jira/browse/RANGER-3456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joseph Aliase updated RANGER-3456:
--
Summary: Improve Kafka Plugin Unauthroized Request Logging (was: Improve
Kafka Plugin Logging )
> Improve Kafka Plugin Unauthroized Request Logging
> --
>
> Key: RANGER-3456
> URL: https://issues.apache.org/jira/browse/RANGER-3456
> Project: Ranger
> Issue Type: Improvement
> Components: plugins
>Affects Versions: 2.1.0
>Reporter: Joseph Aliase
>Priority: Minor
>
> Improve Kafka Plugin:
> It would be helpful if we can log a message when a given IP/session is denied
> access to a resource. Today entire request is logged in debug mode.
>
> Something like this:
> logger.warn("Ip-address ["+ip+"] is not authorized to perform operation
> ["+operation+"] on resource [" + resource+"]")
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Review Request 73617: RANGER-3455 : [Logout-Ranger] Should either be disabled/ should redirect to knox logout page.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73617/ --- Review request for ranger, Dhaval Shah, Dineshkumar Yadav, Jayendra Parab, Kishor Gollapalliwar, Mehul Parikh, Pradeep Agrawal, Sailaja Polavarapu, and Velmurugan Periasamy. Bugs: RANGER-3455 https://issues.apache.org/jira/browse/RANGER-3455 Repository: ranger Description --- Steps: 1. Click on Ranger UI from CP 2. Click on logout button from ranger home page. Observation: For now we see that it lands on below page. But it would be better if we disable logout button /land on knox logout page . Diffs - security-admin/src/main/webapp/scripts/controllers/Controller.js 5fc7e9fcd security-admin/src/main/webapp/scripts/controllers/NController.js 612b0f20b security-admin/src/main/webapp/scripts/views/common/ProfileBar.js 8a8530391 Diff: https://reviews.apache.org/r/73617/diff/1/ Testing --- Tested that after clicking on logout button its land to knox logout page. Thanks, Nitin Galave
[jira] [Updated] (RANGER-3455) [Logout-Ranger] Should either be disabled/ should redirect to knox logout page
[ https://issues.apache.org/jira/browse/RANGER-3455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nitin Galave updated RANGER-3455: - Attachment: 0001-RANGER-3455.patch > [Logout-Ranger] Should either be disabled/ should redirect to knox logout page > -- > > Key: RANGER-3455 > URL: https://issues.apache.org/jira/browse/RANGER-3455 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Nitin Galave >Assignee: Nitin Galave >Priority: Major > Attachments: 0001-RANGER-3455.patch, image-2021-09-23-21-30-04-791.png > > > *Steps:* > 1. Click on Ranger UI from CP > 2. Click on logout button from ranger home page. > *Observation:* > For now we see that it lands on below page. But it would be better if we > disable logout button /land on knox logout page . > !image-2021-09-23-21-30-04-791.png|width=322,height=132! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (RANGER-3455) [Logout-Ranger] Should either be disabled/ should redirect to knox logout page
[ https://issues.apache.org/jira/browse/RANGER-3455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nitin Galave updated RANGER-3455: - Description: *Steps:* 1. Click on Ranger UI from CP 2. Click on logout button from ranger home page. *Observation:* For now we see that it lands on below page. But it would be better if we disable logout button /land on knox logout page . !image-2021-09-23-21-30-04-791.png|width=322,height=132! was: *Steps:* 1. Click on Ranger UI from CP 2. Click on logout button from ranger home page. *Observation:* For now we see that it lands on below page. But it would be better if we disable logout button /land on knox logout page . !image-2021-09-23-21-30-04-791.png! > [Logout-Ranger] Should either be disabled/ should redirect to knox logout page > -- > > Key: RANGER-3455 > URL: https://issues.apache.org/jira/browse/RANGER-3455 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Nitin Galave >Assignee: Nitin Galave >Priority: Major > Attachments: image-2021-09-23-21-30-04-791.png > > > *Steps:* > 1. Click on Ranger UI from CP > 2. Click on logout button from ranger home page. > *Observation:* > For now we see that it lands on below page. But it would be better if we > disable logout button /land on knox logout page . > !image-2021-09-23-21-30-04-791.png|width=322,height=132! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (RANGER-3455) [Logout-Ranger] Should either be disabled/ should redirect to knox logout page
[ https://issues.apache.org/jira/browse/RANGER-3455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nitin Galave updated RANGER-3455: - Attachment: image-2021-09-23-21-30-04-791.png > [Logout-Ranger] Should either be disabled/ should redirect to knox logout page > -- > > Key: RANGER-3455 > URL: https://issues.apache.org/jira/browse/RANGER-3455 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Nitin Galave >Assignee: Nitin Galave >Priority: Major > Attachments: image-2021-09-23-21-30-04-791.png > > > *Steps:* > 1. Click on Ranger UI from CP > 2. Click on logout button from ranger home page. > *Observation:* > For now we see that it lands on below page. But it would be better if we > disable logout button /land on knox logout page . -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (RANGER-3455) [Logout-Ranger] Should either be disabled/ should redirect to knox logout page
[ https://issues.apache.org/jira/browse/RANGER-3455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nitin Galave updated RANGER-3455: - Description: *Steps:* 1. Click on Ranger UI from CP 2. Click on logout button from ranger home page. *Observation:* For now we see that it lands on below page. But it would be better if we disable logout button /land on knox logout page . !image-2021-09-23-21-30-04-791.png! was: *Steps:* 1. Click on Ranger UI from CP 2. Click on logout button from ranger home page. *Observation:* For now we see that it lands on below page. But it would be better if we disable logout button /land on knox logout page . > [Logout-Ranger] Should either be disabled/ should redirect to knox logout page > -- > > Key: RANGER-3455 > URL: https://issues.apache.org/jira/browse/RANGER-3455 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Nitin Galave >Assignee: Nitin Galave >Priority: Major > Attachments: image-2021-09-23-21-30-04-791.png > > > *Steps:* > 1. Click on Ranger UI from CP > 2. Click on logout button from ranger home page. > *Observation:* > For now we see that it lands on below page. But it would be better if we > disable logout button /land on knox logout page . > !image-2021-09-23-21-30-04-791.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (RANGER-3455) [Logout-Ranger] Should either be disabled/ should redirect to knox logout page
Nitin Galave created RANGER-3455: Summary: [Logout-Ranger] Should either be disabled/ should redirect to knox logout page Key: RANGER-3455 URL: https://issues.apache.org/jira/browse/RANGER-3455 Project: Ranger Issue Type: Improvement Components: Ranger Reporter: Nitin Galave Assignee: Nitin Galave *Steps:* 1. Click on Ranger UI from CP 2. Click on logout button from ranger home page. *Observation:* For now we see that it lands on below page. But it would be better if we disable logout button /land on knox logout page . -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (RANGER-3442) Ranger KMS DAO memory issues when many new keys are created
[
https://issues.apache.org/jira/browse/RANGER-3442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17421273#comment-17421273
]
Brahma Reddy Battula edited comment on RANGER-3442 at 9/28/21, 9:28 AM:
[~pavitheran] thanks for reporting and neat analysis. I do see that ranger_KMS
using more memory and need to do profiling (pasting the heapdump here) , not
sure it might to similar you reported but your analysis make sense to me. (
using ranger-0.7.0 with some patches.). In our cluster we've 16 keys, our heap
usage is 60GB+..
How many keys present in your cluster..?
{noformat}
num #instances #bytes class name
--
1: 12514991 1125037584 [C
2: 3194026 629310896 [Ljava.util.HashMap$Node;
3: 4547927 421361800 [B
4: 12492061 399745952 java.lang.String
5: 7398577 355131696 java.util.HashMap$Node
6: 3189692 280692896 org.apache.zookeeper.data.Stat
7: 3197676 204651264 java.util.HashMap
8: 3207962 153982176 java.util.concurrent.ConcurrentHashMap$Node
9: 3189693 127587720
org.apache.curator.framework.recipes.cache.ChildData
10: 3190260 76566240 java.util.HashSet
11: 3189714 76553136 java.util.concurrent.atomic.AtomicReference
12: 1349 67582872
[Ljava.util.concurrent.ConcurrentHashMap$Node;
13: 1263345 40427040 org.apache.hadoop.io.Text
14: 23853 35543088 [Ljava.lang.Object;
15: 421113 33689040
org.apache.hadoop.security.token.delegation.web.DelegationTokenIdentifier
16: 421113 16844520
org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager$DelegationTokenInformation
num #instances #bytes class name
--
1: 12494150 1126339552 [C
2: 3189356 561492704 [Ljava.util.HashMap$Node;
3: 4627687 429962248 [B
4: 12470326 399050432 java.lang.String
5: 6811252 326940096 java.util.HashMap$Node
6: 3186259 280390792 org.apache.zookeeper.data.Stat
7: 3194119 204423616 java.util.HashMap
8: 3204325 153807600 java.util.concurrent.ConcurrentHashMap$Node
9: 3186257 127450280
org.apache.curator.framework.recipes.cache.ChildData
10: 3186819 76483656 java.util.HashSet
11: 3186279 76470696 java.util.concurrent.atomic.AtomicReference
12: 1420 67593664
[Ljava.util.concurrent.ConcurrentHashMap$Node;
13: 96175 43101976 [Ljava.lang.Object;
14: 1253046 40097472 org.apache.hadoop.io.Text
15: 417680 33414400
org.apache.hadoop.security.token.delegation.web.DelegationTokenIdentifier
16: 417680 16707200
org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager$DelegationTokenInformation
17: 79595 14008720 java.net.SocksSocketImpl
18: 112616 7207424 java.lang.ref.Finalizer
19: 79591 7004008 java.net.SocketInputStream
20: 79591 7004008 java.net.SocketOutputStream
21: 414599 6633584 java.lang.Object
22: 79101 5695272 org.apache.tomcat.util.buf.ByteChunk
23: 67540 4322560 org.apache.tomcat.util.buf.CharChunk
24: 61120 3911680 org.apache.tomcat.util.buf.MessageBytes
25: 39328 3885784 [Ljava.lang.String;
26: 84221 3368840 java.util.ArrayList
27: 79610 3184400 java.io.FileDescriptor
{noformat}
was (Author: brahmareddy):
[~pavitheran] thanks for reporting and neat analysis. I do see that ranger_KMS
using more memory and need to do profiling (pasting the heapdump here) , not
sure it might to similar you reported but your analysis make sense to me. (
using ranger-0.7.0 with some patches.)
{noformat}
num #instances #bytes class name
--
1: 12514991 1125037584 [C
2: 3194026 629310896 [Ljava.util.HashMap$Node;
3: 4547927 421361800 [B
4: 12492061 399745952 java.lang.String
5: 7398577 355131696 java.util.HashMap$Node
6: 3189692 280692896 org.apache.zookeeper.data.Stat
7: 3197676 204651264 java.util.HashMap
8: 3207962 153982176 java.util.concurrent.ConcurrentHashMap$Node
9: 3189693 127587720
org.apache.curator.framework.recipes.cache.ChildData
10: 3190260 76566240 java.util.HashSet
11: 3189714 76553136 java.util.concurrent.atomic.AtomicReference
12: 1349
[jira] [Commented] (RANGER-3442) Ranger KMS DAO memory issues when many new keys are created
[
https://issues.apache.org/jira/browse/RANGER-3442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17421273#comment-17421273
]
Brahma Reddy Battula commented on RANGER-3442:
--
[~pavitheran] thanks for reporting and neat analysis. I do see that ranger_KMS
using more memory and need to do profiling (pasting the heapdump here) , not
sure it might to similar you reported but your analysis make sense to me. (
using ranger-0.7.0 with some patches.)
{noformat}
num #instances #bytes class name
--
1: 12514991 1125037584 [C
2: 3194026 629310896 [Ljava.util.HashMap$Node;
3: 4547927 421361800 [B
4: 12492061 399745952 java.lang.String
5: 7398577 355131696 java.util.HashMap$Node
6: 3189692 280692896 org.apache.zookeeper.data.Stat
7: 3197676 204651264 java.util.HashMap
8: 3207962 153982176 java.util.concurrent.ConcurrentHashMap$Node
9: 3189693 127587720
org.apache.curator.framework.recipes.cache.ChildData
10: 3190260 76566240 java.util.HashSet
11: 3189714 76553136 java.util.concurrent.atomic.AtomicReference
12: 1349 67582872
[Ljava.util.concurrent.ConcurrentHashMap$Node;
13: 1263345 40427040 org.apache.hadoop.io.Text
14: 23853 35543088 [Ljava.lang.Object;
15: 421113 33689040
org.apache.hadoop.security.token.delegation.web.DelegationTokenIdentifier
16: 421113 16844520
org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager$DelegationTokenInformation
num #instances #bytes class name
--
1: 12494150 1126339552 [C
2: 3189356 561492704 [Ljava.util.HashMap$Node;
3: 4627687 429962248 [B
4: 12470326 399050432 java.lang.String
5: 6811252 326940096 java.util.HashMap$Node
6: 3186259 280390792 org.apache.zookeeper.data.Stat
7: 3194119 204423616 java.util.HashMap
8: 3204325 153807600 java.util.concurrent.ConcurrentHashMap$Node
9: 3186257 127450280
org.apache.curator.framework.recipes.cache.ChildData
10: 3186819 76483656 java.util.HashSet
11: 3186279 76470696 java.util.concurrent.atomic.AtomicReference
12: 1420 67593664
[Ljava.util.concurrent.ConcurrentHashMap$Node;
13: 96175 43101976 [Ljava.lang.Object;
14: 1253046 40097472 org.apache.hadoop.io.Text
15: 417680 33414400
org.apache.hadoop.security.token.delegation.web.DelegationTokenIdentifier
16: 417680 16707200
org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager$DelegationTokenInformation
17: 79595 14008720 java.net.SocksSocketImpl
18: 112616 7207424 java.lang.ref.Finalizer
19: 79591 7004008 java.net.SocketInputStream
20: 79591 7004008 java.net.SocketOutputStream
21: 414599 6633584 java.lang.Object
22: 79101 5695272 org.apache.tomcat.util.buf.ByteChunk
23: 67540 4322560 org.apache.tomcat.util.buf.CharChunk
24: 61120 3911680 org.apache.tomcat.util.buf.MessageBytes
25: 39328 3885784 [Ljava.lang.String;
26: 84221 3368840 java.util.ArrayList
27: 79610 3184400 java.io.FileDescriptor
{noformat}
> Ranger KMS DAO memory issues when many new keys are created
> ---
>
> Key: RANGER-3442
> URL: https://issues.apache.org/jira/browse/RANGER-3442
> Project: Ranger
> Issue Type: Bug
> Components: kms
>Affects Versions: 2.0.0
>Reporter: Pavi Subenderan
>Priority: Major
>
> We have many keys created in our KMS keystore and recently we found that when
> we create new keys, the KMS instances easily hit against the memory limit.
> We can reproduce this with a script to call KMS createKey and then
> getMetadata for new keys in a loop. Basically we restart our instances and
> memory usage is approximately 1.5GB out of 8GB, but after running this script
> for a bit (1-5 minutes), we hit close to the 8GB limit and the memory usage
> does not go back down after that.
> I did a heap dump and saw that most of the memory was being retained by
> XXRangerKeystore and eclipse EntityManagerImpl.
> * org.eclipse.persistence.internal.jpa.EntityManagerImpl
> * org.eclipse.persistence.internal.sessions.RepeatableWriteUnitOfWork
> And the largest shallow size object was char[] with 4GB+...
>
> *My fix*
Permissions to edit "Want to Contribute to Apache Ranger?" wiki page
Hello Ranger devs, Could you please give me permissions to edit https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=55151244 ? I'd like to fix the url to GitBox (ex. git-wip-us) and add few more details about the Pull Requests at GitHub. My apache id is: mgrigorov Thank you! Regards, Martin
