[jira] [Updated] (RANGER-4800) Ranger Tagsync service start failed when ranger admin is SSL enabled
[
https://issues.apache.org/jira/browse/RANGER-4800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bhavik Patel updated RANGER-4800:
-
Attachment: 0001-RANGER-4800-Ranger-Tagsync-service-start-failed-when.patch
> Ranger Tagsync service start failed when ranger admin is SSL enabled
>
>
> Key: RANGER-4800
> URL: https://issues.apache.org/jira/browse/RANGER-4800
> Project: Ranger
> Issue Type: Bug
> Components: tagsync
>Affects Versions: 2.4.0
>Reporter: Bhavik Patel
>Assignee: Bhavik Patel
>Priority: Major
> Attachments:
> 0001-RANGER-4800-Ranger-Tagsync-service-start-failed-when.patch
>
>
> Ranger Tagsync service start failed when ranger admin is SSL enabled with
> below error:
> {code:java}
> ERROR o.a.r.t.p.TagSynchronizer [main] - 230 Failed to initialize TAG sink.
> Error details:
> java.lang.NoClassDefFoundError:
> org/apache/ranger/authorization/hadoop/utils/RangerCredentialProvider
> at
> org.apache.ranger.plugin.util.RangerRESTClient.getCredential(RangerRESTClient.java:433)
> at
> org.apache.ranger.plugin.util.RangerRESTClient.getKeyManagers(RangerRESTClient.java:291)
> at
> org.apache.ranger.plugin.util.RangerRESTClient.buildClient(RangerRESTClient.java:205)
> at
> org.apache.ranger.plugin.util.RangerRESTClient.getClient(RangerRESTClient.java:193)
> at
> org.apache.ranger.tagsync.sink.tagadmin.TagAdminRESTSink.initialize(TagAdminRESTSink.java:106)
> at
> org.apache.ranger.tagsync.process.TagSynchronizer.initializeTagSink(TagSynchronizer.java:225)
> at
> org.apache.ranger.tagsync.process.TagSynchronizer.initialize(TagSynchronizer.java:110)
> at
> org.apache.ranger.tagsync.process.TagSynchronizer.main(TagSynchronizer.java:65)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.ranger.authorization.hadoop.utils.RangerCredentialProvider
> at
> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
> at
> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
> ... 8 common frames omitted
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (RANGER-4800) Ranger Tagsync service start failed when ranger admin is SSL enabled
Bhavik Patel created RANGER-4800:
Summary: Ranger Tagsync service start failed when ranger admin is
SSL enabled
Key: RANGER-4800
URL: https://issues.apache.org/jira/browse/RANGER-4800
Project: Ranger
Issue Type: Bug
Components: tagsync
Affects Versions: 2.4.0
Reporter: Bhavik Patel
Assignee: Bhavik Patel
Ranger Tagsync service start failed when ranger admin is SSL enabled with below
error:
{code:java}
ERROR o.a.r.t.p.TagSynchronizer [main] - 230 Failed to initialize TAG sink.
Error details:
java.lang.NoClassDefFoundError:
org/apache/ranger/authorization/hadoop/utils/RangerCredentialProvider
at
org.apache.ranger.plugin.util.RangerRESTClient.getCredential(RangerRESTClient.java:433)
at
org.apache.ranger.plugin.util.RangerRESTClient.getKeyManagers(RangerRESTClient.java:291)
at
org.apache.ranger.plugin.util.RangerRESTClient.buildClient(RangerRESTClient.java:205)
at
org.apache.ranger.plugin.util.RangerRESTClient.getClient(RangerRESTClient.java:193)
at
org.apache.ranger.tagsync.sink.tagadmin.TagAdminRESTSink.initialize(TagAdminRESTSink.java:106)
at
org.apache.ranger.tagsync.process.TagSynchronizer.initializeTagSink(TagSynchronizer.java:225)
at
org.apache.ranger.tagsync.process.TagSynchronizer.initialize(TagSynchronizer.java:110)
at
org.apache.ranger.tagsync.process.TagSynchronizer.main(TagSynchronizer.java:65)
Caused by: java.lang.ClassNotFoundException:
org.apache.ranger.authorization.hadoop.utils.RangerCredentialProvider
at
java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
at
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
... 8 common frames omitted
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (RANGER-3409) Update Jackson and remove Codehaus version
[ https://issues.apache.org/jira/browse/RANGER-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848127#comment-17848127 ] Bhavik Patel commented on RANGER-3409: -- [~sercan.tekin] I have reviewed your PR but i can see still the usage of "codehaus" package. > Update Jackson and remove Codehaus version > -- > > Key: RANGER-3409 > URL: https://issues.apache.org/jira/browse/RANGER-3409 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Affects Versions: 3.0.0 >Reporter: Andrew Charneski >Priority: Blocker > Attachments: ranger-modify-policy.png > > Time Spent: 0.5h > Remaining Estimate: 0h > > An old version of Jackson (Codehaus Jackson 1.9.13) is still being used. > Jackson has since moved namespaces with a reorganized library structure. > Update all references to the older version to use the newer version (which is > currently used in some modules). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-3409) Update Jackson and remove Codehaus version
[ https://issues.apache.org/jira/browse/RANGER-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846617#comment-17846617 ] Bhavik Patel commented on RANGER-3409: -- Correct [~sercan.tekin]. Just curious to know how it is working for [~n.blagodarny] . > Update Jackson and remove Codehaus version > -- > > Key: RANGER-3409 > URL: https://issues.apache.org/jira/browse/RANGER-3409 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Affects Versions: 3.0.0 >Reporter: Andrew Charneski >Priority: Blocker > Time Spent: 0.5h > Remaining Estimate: 0h > > An old version of Jackson (Codehaus Jackson 1.9.13) is still being used. > Jackson has since moved namespaces with a reorganized library structure. > Update all references to the older version to use the newer version (which is > currently used in some modules). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-3409) Update Jackson and remove Codehaus version
[ https://issues.apache.org/jira/browse/RANGER-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846588#comment-17846588 ] Bhavik Patel commented on RANGER-3409: -- Thanks [~n.blagodarny] . While validating your changes I observed users are not visible in the ranger admin web ui. After debugging found that while migrating from Codehaus Jackson to FasterXML Jackson property naming strategy was changed to lowercase and on the frontend we are expecting the original object(without any conversion). Just wondering how does this worked for you. > Update Jackson and remove Codehaus version > -- > > Key: RANGER-3409 > URL: https://issues.apache.org/jira/browse/RANGER-3409 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Affects Versions: 3.0.0 >Reporter: Andrew Charneski >Priority: Blocker > Time Spent: 0.5h > Remaining Estimate: 0h > > An old version of Jackson (Codehaus Jackson 1.9.13) is still being used. > Jackson has since moved namespaces with a reorganized library structure. > Update all references to the older version to use the newer version (which is > currently used in some modules). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4225) Possible Jackson serialization issue due to not comply with Java bean standards
[
https://issues.apache.org/jira/browse/RANGER-4225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843650#comment-17843650
]
Bhavik Patel commented on RANGER-4225:
--
[~sercan.tekin] are you working on this?
> Possible Jackson serialization issue due to not comply with Java bean
> standards
> ---
>
> Key: RANGER-4225
> URL: https://issues.apache.org/jira/browse/RANGER-4225
> Project: Ranger
> Issue Type: Bug
> Components: Ranger
>Affects Versions: 2.4.0
>Reporter: Sercan Tekin
>Assignee: Sercan Tekin
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: RANGER-4225.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> +*PROBLEM:*+
> Transitive Jackson-2 dependencies are included into Ranger's classpath in my
> env and conflicted with Jackson-1 dependencies.
> Jackson-2 uses Javabean naming conventions to figure out the Json properties
> in a Java class, and some of the Ranger's model classes don't comply with the
> convention.
> For example, when the leading camelcase word is only one letter in length,
> then deserialized response is broken. The following is what I observed in
> Ranger v2.4.0;
> On Ranger UI side, this
> [code-block|https://github.com/apache/ranger/blob/ranger-1.2/security-admin/src/main/webapp/scripts/views/policies/PermissionList.js#L224-L229]
> attempts to read vXStrings key in map, but the corresponding response has
> vxstrings instead:
> {code:java}
> {
> "startIndex": 0,
> "pageSize": 200,
> "totalCount": 11,
> "resultSize": 11,
> "sortType": "asc",
> "sortBy": "id",
> "listSize": 11,
> "vxstrings": [< here! This has to be vXStrings
> {
> "value": "public",
> ... {code}
> And this difference causes below error while reading the property, therefore
> the corresponding dropdown has no values as excepted;
> {code:java}
> PermissionList.js?ver=build.version:226 Uncaught TypeError: Cannot read
> properties of undefined (reading 'map')
> at Object.results (PermissionList.js?ver=build.version:226:34)
> at Object.success (select2.js?ver=build.version:450:47)
> at u (jquery-3.3.1.min.js?ver=build.version:2:27457)
> at Object.fireWith [as resolveWith]
> (jquery-3.3.1.min.js?ver=build.version:2:28202)
> at k (jquery-3.3.1.min.js?ver=build.version:2:77651)
> at XMLHttpRequest.
> (jquery-3.3.1.min.js?ver=build.version:2:79907){code}
> +*REFERENCES:*+
> Please see this reference related to capital letters
> [http://futuretask.blogspot.com/2005/01/java-tip-6-dont-capitalize-first-two.html]
> "Don't capitalize first two letters of a bean property name. This is in our
> java standards. You should not create a java bean property name that begins
> with a capital letter in the 1st two places."
> Also you can see the same issue is reported here
> [https://stackoverflow.com/questions/30205006/why-does-jackson-2-not-recognize-the-first-capital-letter-if-the-leading-camel-c]
>
> +*SOLUTION:*+
> {{@JsonProperty}} annotation needs to be added for mapping the properties
> with their corresponding getter/setter methods. This will not effect Ranger's
> functionality directly, but it will provide consistency even if Jackson-2 is
> included into classpath.
> I have tested it locally after adding {{@JsonProperty}} and everything worked
> well.
> The PR is attached.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (RANGER-4622) Fix the default value of ranger.accesslog.rotate.rename_on_rotate configuration in ranger--admin-default-site.xml
[ https://issues.apache.org/jira/browse/RANGER-4622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17832752#comment-17832752 ] Bhavik Patel commented on RANGER-4622: -- +LGTM > Fix the default value of ranger.accesslog.rotate.rename_on_rotate > configuration in ranger--admin-default-site.xml > - > > Key: RANGER-4622 > URL: https://issues.apache.org/jira/browse/RANGER-4622 > Project: Ranger > Issue Type: Improvement > Components: admin >Affects Versions: 2.3.0 >Reporter: zhaoshuaihua >Priority: Minor > Attachments: > RANGER-4622_property_ranger_accesslog_rotate_rename_on_rotate,_the_default_value_is_false,.patch, > screenshot-1.png > > Original Estimate: 0.5h > Time Spent: 20m > Remaining Estimate: 10m > > Fix the default value of ranger.accesslog.rotate.rename_on_rotate > configuration in ranger--admin-default-site.xml. The default value of this > configuration is false instead of 15. > > !https://private-user-images.githubusercontent.com/50791733/291463576-4dbadd0b-0c89-411f-84bf-9f173dcdfd4c.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTEiLCJleHAiOjE3MDMxNDczMzIsIm5iZiI6MTcwMzE0NzAzMiwicGF0aCI6Ii81MDc5MTczMy8yOTE0NjM1NzYtNGRiYWRkMGItMGM4OS00MTFmLTg0YmYtOWYxNzNkY2RmZDRjLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFJV05KWUFYNENTVkVINTNBJTJGMjAyMzEyMjElMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjMxMjIxVDA4MjM1MlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPThiYTkwZGJhYmM5NWM5YWVlYzhkMTgyYmIyMjhhYzkwMzg3MTUwZmU3YzQ5MTcxMzY1OGQyZWRiMzhiNDEyYzImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.KjaxtRsqR1N9JuBgx4n3_45_tzIa9liHl787PCtqUFo! > !https://private-user-images.githubusercontent.com/50791733/291463606-a72f6e18-fcee-4f8d-8155-fa7cb9eb76e4.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTEiLCJleHAiOjE3MDMxNDczMzIsIm5iZiI6MTcwMzE0NzAzMiwicGF0aCI6Ii81MDc5MTczMy8yOTE0NjM2MDYtYTcyZjZlMTgtZmNlZS00ZjhkLTgxNTUtZmE3Y2I5ZWI3NmU0LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFJV05KWUFYNENTVkVINTNBJTJGMjAyMzEyMjElMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjMxMjIxVDA4MjM1MlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTc3YTg5YjhjZmIxNzUwYjkyYzE1MzBhZjNmOGEwZjI4NGZmODEzMDg5NDQ2M2VkNTU5YmVkNGEyNWEwNDc1M2QmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.k5_0FMEOriQX_Q0_hd7vx5LmOR6p3PV8ozwKQ1p9hv0! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4511) Enable Parallel Compilation for Multiple Modules in Ranger
[ https://issues.apache.org/jira/browse/RANGER-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823437#comment-17823437 ] Bhavik Patel commented on RANGER-4511: -- [~jialiang] This would be great if have parallel compilation. [~madhan] Can you also check once. > Enable Parallel Compilation for Multiple Modules in Ranger > -- > > Key: RANGER-4511 > URL: https://issues.apache.org/jira/browse/RANGER-4511 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: caijialiang >Assignee: caijialiang >Priority: Major > Attachments: image-2023-11-03-11-15-50-593.png, > patch1-RANGER-4511.diff > > Time Spent: 10m > Remaining Estimate: 0h > > "Due to incorrect dependency settings in the distro module, Ranger cannot > utilize multicore concurrent compilation. As a result, compiling Ranger is > very slow each time, as all modules can only be compiled serially. > With this patch, Ranger can be compiled in parallel, greatly speeding up the > process. Compilation can now be completed in just a few minutes. > !image-2023-11-03-11-15-50-593.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4735) Ranger KMS Unable to Start - Missing hadoop-shade-guava JAR
[ https://issues.apache.org/jira/browse/RANGER-4735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823435#comment-17823435 ] Bhavik Patel commented on RANGER-4735: -- hadoop version upgrade jira: https://issues.apache.org/jira/browse/RANGER-4418 > Ranger KMS Unable to Start - Missing hadoop-shade-guava JAR > --- > > Key: RANGER-4735 > URL: https://issues.apache.org/jira/browse/RANGER-4735 > Project: Ranger > Issue Type: Bug > Components: admin >Affects Versions: 2.4.0 >Reporter: caijialiang >Assignee: caijialiang >Priority: Major > Attachments: image-2024-03-05-09-34-48-812.png > > > Ranger Kms can't start due to lack hadoop-shade-guava jar > !image-2024-03-05-09-34-48-812.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4735) Ranger KMS Unable to Start - Missing hadoop-shade-guava JAR
[ https://issues.apache.org/jira/browse/RANGER-4735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823428#comment-17823428 ] Bhavik Patel commented on RANGER-4735: -- Then this is expected, There is already open jira for hadoop version upgrade. > Ranger KMS Unable to Start - Missing hadoop-shade-guava JAR > --- > > Key: RANGER-4735 > URL: https://issues.apache.org/jira/browse/RANGER-4735 > Project: Ranger > Issue Type: Bug > Components: admin >Affects Versions: 2.4.0 >Reporter: caijialiang >Assignee: caijialiang >Priority: Major > Attachments: image-2024-03-05-09-34-48-812.png > > > Ranger Kms can't start due to lack hadoop-shade-guava jar > !image-2024-03-05-09-34-48-812.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4076) Support Java 17 for build and runtime
[ https://issues.apache.org/jira/browse/RANGER-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823416#comment-17823416 ] Bhavik Patel commented on RANGER-4076: -- [~pradeep] With the current attached patch we are able to build the ranger but we have to also upgrade the hadoop and other component version which supports the JDK-17. With the current patch I'm expecting some issue might observed during/after the ranger KMS upgrade. > Support Java 17 for build and runtime > - > > Key: RANGER-4076 > URL: https://issues.apache.org/jira/browse/RANGER-4076 > Project: Ranger > Issue Type: New Feature > Components: admin, build-infra >Reporter: Andrew Luo >Priority: Major > Attachments: > 0001-RANGER-4076-Support-Java-17-for-build-and-runtime.patch > > > Currently only Java 8 and 11 are supported. Java 17 is a major LTS version > of Java and adding support would modernize our Java version support. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4476) trino filter
[
https://issues.apache.org/jira/browse/RANGER-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823413#comment-17823413
]
Bhavik Patel commented on RANGER-4476:
--
Yes, [~pradeep] We required to do code changes for JDK-17.
There are multiple jira's for trino plugin support and multiple PR's are also
opened. Let's have one common jira and one final PR.
> trino filter
>
>
> Key: RANGER-4476
> URL: https://issues.apache.org/jira/browse/RANGER-4476
> Project: Ranger
> Issue Type: Bug
> Components: plugins, Ranger
>Affects Versions: 3.0.0
>Reporter: Andrey Pilipyuk
>Priority: Major
> Labels: Trino
>
> hello there
> i trying to enable trino plugin, i have successful connect check
> but when i try to make policy i have an error in logs
> 2023-10-16 13:39:09,055 [https-jsse-nio-6182-exec-6] WARN
> [ServiceMgr.java:476] getFilesInDirectory('ranger-plugins/trino'): adding
> /opt/ranger-admin-3.0.0/ews/webapp/WEB-INF/classes/ranger-plugins/trino/trino-jdbc-423.jar
> 2023-10-16 13:39:09,055 [https-jsse-nio-6182-exec-6] WARN
> [ServiceMgr.java:476] getFilesInDirectory('ranger-plugins/trino'): adding
> /opt/ranger-admin-3.0.0/ews/webapp/WEB-INF/classes/ranger-plugins/trino/ranger-trino-plugin-3.0.0-SNAPSHOT.jar
> 2023-10-16 13:39:09,056 [https-jsse-nio-6182-exec-6] WARN
> [ServiceMgr.java:476] getFilesInDirectory('ranger-plugins/trino'): adding
> /opt/ranger-admin-3.0.0/ews/webapp/WEB-INF/classes/ranger-plugins/trino/trino-spi-423.jar
> 2023-10-16 13:39:09,078 [timed-executor-pool-0] WARN
> [SecureClientLogin.java:123] Can't find keyTab Path : null
> 2023-10-16 13:39:09,079 [timed-executor-pool-0] WARN
> [SecureClientLogin.java:127] Can't find principal : null
> 2023-10-16 13:39:09,084 [timed-executor-pool-0] INFO [BaseClient.java:126]
> Init Login: security not enabled, using username
> 2023-10-16 13:39:09,501 [timed-executor-pool-0] ERROR
> [TrinoResourceManager.java:168] Could not initiate a TrinoClient timedTask
> can you please tell me how i can debug it, or some prefirences to trino from
> ranger
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (RANGER-4735) Ranger KMS Unable to Start - Missing hadoop-shade-guava JAR
[ https://issues.apache.org/jira/browse/RANGER-4735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823412#comment-17823412 ] Bhavik Patel commented on RANGER-4735: -- Which version of hadoop you are using? > Ranger KMS Unable to Start - Missing hadoop-shade-guava JAR > --- > > Key: RANGER-4735 > URL: https://issues.apache.org/jira/browse/RANGER-4735 > Project: Ranger > Issue Type: Bug > Components: admin >Affects Versions: 2.4.0 >Reporter: caijialiang >Assignee: caijialiang >Priority: Major > Attachments: image-2024-03-05-09-34-48-812.png > > > Ranger Kms can't start due to lack hadoop-shade-guava jar > !image-2024-03-05-09-34-48-812.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4076) Support Java 17 for build and runtime
[ https://issues.apache.org/jira/browse/RANGER-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17813046#comment-17813046 ] Bhavik Patel commented on RANGER-4076: -- [~rakeshgupta264] have you validated rangeradmin , usersync, tagsync and KMS functionality with JDK-17? > Support Java 17 for build and runtime > - > > Key: RANGER-4076 > URL: https://issues.apache.org/jira/browse/RANGER-4076 > Project: Ranger > Issue Type: New Feature > Components: admin, build-infra >Reporter: Andrew Luo >Priority: Major > Attachments: > 0001-RANGER-4076-Support-Java-17-for-build-and-runtime.patch > > > Currently only Java 8 and 11 are supported. Java 17 is a major LTS version > of Java and adding support would modernize our Java version support. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4667) Replace Nashron script engine with GraalVM
[ https://issues.apache.org/jira/browse/RANGER-4667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809333#comment-17809333 ] Bhavik Patel commented on RANGER-4667: -- Yes, [~madhan] I reviewed [RANGER-3970|https://issues.apache.org/jira/browse/RANGER-3970] and [RANGER-4401|https://issues.apache.org/jira/browse/RANGER-4401]. > Replace Nashron script engine with GraalVM > -- > > Key: RANGER-4667 > URL: https://issues.apache.org/jira/browse/RANGER-4667 > Project: Ranger > Issue Type: Improvement > Components: plugins, Ranger >Reporter: Bhavik Patel >Priority: Major > > Replace Nashron scrip engine with default GraalVM engine this will help in > support for JDK-17 > cc: [~madhan] / [~rdonbosco] / [~sneethir] / [~kishor.gollapalliwar] / > [~bosco] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-4667) Replace Nashron scrip engine with GraalVM
[ https://issues.apache.org/jira/browse/RANGER-4667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavik Patel updated RANGER-4667: - Description: Replace Nashron scrip engine with default GraalVM engine this will help in support for JDK-17 cc: [~madhan] / [~rdonbosco] / [~sneethir] / [~kishor.gollapalliwar] / [~bosco] was: Replace Nashron scrip engine with default GraalVM engine this will help in support for JDK-17 cc: [~madhan] / [~rdonbosco] / [~sneethir] > Replace Nashron scrip engine with GraalVM > - > > Key: RANGER-4667 > URL: https://issues.apache.org/jira/browse/RANGER-4667 > Project: Ranger > Issue Type: Improvement > Components: plugins, Ranger >Reporter: Bhavik Patel >Priority: Major > > Replace Nashron scrip engine with default GraalVM engine this will help in > support for JDK-17 > cc: [~madhan] / [~rdonbosco] / [~sneethir] / [~kishor.gollapalliwar] / > [~bosco] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-4667) Replace Nashron script engine with GraalVM
[ https://issues.apache.org/jira/browse/RANGER-4667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavik Patel updated RANGER-4667: - Summary: Replace Nashron script engine with GraalVM (was: Replace Nashron scrip engine with GraalVM) > Replace Nashron script engine with GraalVM > -- > > Key: RANGER-4667 > URL: https://issues.apache.org/jira/browse/RANGER-4667 > Project: Ranger > Issue Type: Improvement > Components: plugins, Ranger >Reporter: Bhavik Patel >Priority: Major > > Replace Nashron scrip engine with default GraalVM engine this will help in > support for JDK-17 > cc: [~madhan] / [~rdonbosco] / [~sneethir] / [~kishor.gollapalliwar] / > [~bosco] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4667) Replace Nashron scrip engine with GraalVM
Bhavik Patel created RANGER-4667: Summary: Replace Nashron scrip engine with GraalVM Key: RANGER-4667 URL: https://issues.apache.org/jira/browse/RANGER-4667 Project: Ranger Issue Type: Improvement Components: plugins, Ranger Reporter: Bhavik Patel Replace Nashron scrip engine with default GraalVM engine this will help in support for JDK-17 cc: [~madhan] / [~rdonbosco] / [~sneethir] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4225) Possible Jackson serialization issue due to not comply with Java bean standards
[
https://issues.apache.org/jira/browse/RANGER-4225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807051#comment-17807051
]
Bhavik Patel commented on RANGER-4225:
--
Sure.
Thanks [~sercan.tekin].
> Possible Jackson serialization issue due to not comply with Java bean
> standards
> ---
>
> Key: RANGER-4225
> URL: https://issues.apache.org/jira/browse/RANGER-4225
> Project: Ranger
> Issue Type: Bug
> Components: Ranger
>Affects Versions: 2.4.0
>Reporter: Sercan Tekin
>Assignee: Sercan Tekin
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: RANGER-4225.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> +*PROBLEM:*+
> Transitive Jackson-2 dependencies are included into Ranger's classpath in my
> env and conflicted with Jackson-1 dependencies.
> Jackson-2 uses Javabean naming conventions to figure out the Json properties
> in a Java class, and some of the Ranger's model classes don't comply with the
> convention.
> For example, when the leading camelcase word is only one letter in length,
> then deserialized response is broken. The following is what I observed in
> Ranger v2.4.0;
> On Ranger UI side, this
> [code-block|https://github.com/apache/ranger/blob/ranger-1.2/security-admin/src/main/webapp/scripts/views/policies/PermissionList.js#L224-L229]
> attempts to read vXStrings key in map, but the corresponding response has
> vxstrings instead:
> {code:java}
> {
> "startIndex": 0,
> "pageSize": 200,
> "totalCount": 11,
> "resultSize": 11,
> "sortType": "asc",
> "sortBy": "id",
> "listSize": 11,
> "vxstrings": [< here! This has to be vXStrings
> {
> "value": "public",
> ... {code}
> And this difference causes below error while reading the property, therefore
> the corresponding dropdown has no values as excepted;
> {code:java}
> PermissionList.js?ver=build.version:226 Uncaught TypeError: Cannot read
> properties of undefined (reading 'map')
> at Object.results (PermissionList.js?ver=build.version:226:34)
> at Object.success (select2.js?ver=build.version:450:47)
> at u (jquery-3.3.1.min.js?ver=build.version:2:27457)
> at Object.fireWith [as resolveWith]
> (jquery-3.3.1.min.js?ver=build.version:2:28202)
> at k (jquery-3.3.1.min.js?ver=build.version:2:77651)
> at XMLHttpRequest.
> (jquery-3.3.1.min.js?ver=build.version:2:79907){code}
> +*REFERENCES:*+
> Please see this reference related to capital letters
> [http://futuretask.blogspot.com/2005/01/java-tip-6-dont-capitalize-first-two.html]
> "Don't capitalize first two letters of a bean property name. This is in our
> java standards. You should not create a java bean property name that begins
> with a capital letter in the 1st two places."
> Also you can see the same issue is reported here
> [https://stackoverflow.com/questions/30205006/why-does-jackson-2-not-recognize-the-first-capital-letter-if-the-leading-camel-c]
>
> +*SOLUTION:*+
> {{@JsonProperty}} annotation needs to be added for mapping the properties
> with their corresponding getter/setter methods. This will not effect Ranger's
> functionality directly, but it will provide consistency even if Jackson-2 is
> included into classpath.
> I have tested it locally after adding {{@JsonProperty}} and everything worked
> well.
> The PR is attached.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (RANGER-4326) Cannot renew token when multiple KMS are applied.
[
https://issues.apache.org/jira/browse/RANGER-4326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bhavik Patel resolved RANGER-4326.
--
Resolution: Not A Problem
> Cannot renew token when multiple KMS are applied.
> -
>
> Key: RANGER-4326
> URL: https://issues.apache.org/jira/browse/RANGER-4326
> Project: Ranger
> Issue Type: Bug
> Components: kms
>Affects Versions: 2.3.0, 2.4.0
>Reporter: kirby zhou
>Priority: Major
>
> When multiple KMS are applied with kerberos. Flink on yarn can not renew
> tokens.
>
> Flink calls FileSystem.addDelegationTokens to get all tokens to renew.
> FileSystem.addDelegationTokens calls collectDelegationTokens to collect all
> tokens.
> When it calls LoadBalancingKMSClientProvider.getDelegationToken.
> LoadBalancingKMSClientProvider calls doOp to call one of N
> KMSClientProvider.getDelegationToken().
>
> When renew the token, LoadBalancingKMSClientProvider may call another
> KMSClientProvider to do op. It usually fails.
>
> FYI: have already set hadoop.kms.authentication.signer.secret.provider=file,
> and hadoop.kms.authentication.signature.secret.file="same content file".
>
> Some Sample code:
> {code:java}
> public static void main(String[] args) throws Exception {
> Configuration conf = new Configuration();
> conf.set("hadoop.security.authorization", "true");
> conf.set("hadoop.security.authentication", "kerberos");
> conf.set("dfs.data.transfer.protection", "authentication");
> conf.set("hadoop.security.key.provider.path",
> "kms://http@kms01;kms02:9292/kms");
> conf.set("dfs.client.ignore.namenode.default.kms.uri", "true");
> conf.set("fs.defaultFS", "hdfs://namenode");
> // Login with keytab
> UserGroupInformation.setConfiguration(conf);
> UserGroupInformation.loginUserFromKeytab("testuser@TESTREALM",
> "/Users/kirbyzhou/Develop/testuser.keytab");
> UserGroupInformation ugi = UserGroupInformation.getCurrentUser();
> System.out.println(UserGroupInformation.getCurrentUser().getUserName());
> // GetFS
> FileSystem fs = FileSystem.get(conf);
>
> System.out.println(((DistributedFileSystem)fs).getClient().getKeyProviderUri());
> // Renew
> for (int i = 0; i < 20; ++i) {
> Thread.sleep(200);
> System.out.printf("===pass %02d===\n", i);
> {
> System.out.println("==begin renew==");
> Credentials credentials = ugi.getCredentials();
> fs.addDelegationTokens("sa_cluster", credentials);
> for (Token token : credentials.getAllTokens()) {
> System.out.println(token);
> try {
> token.renew(conf);
> } catch (IOException e) {
> System.err.println(e);
> }
> }
> System.out.println("==end renew==");
> }
> }
> }
> {code}
> A lot of exceptions happens
> {code:java}
> ava.io.IOException: HTTP status [403], message [Forbidden], URL
> [http://kms01:9292/kms/v1/?op=RENEWDELEGATIONTOKEN=KgAKc2FfY2x1c3RlcgpzYV9jbHVzdGVyAIoBiYffA4WKAYmr64eFjgG_AhQ7Oo9G0Lc8IguxB0IgenAHsJ--DQZrbXMtZHRPa21zOi8vaHR0cEBrbXMwMS10aHJvbmUwMS5zZW5zb3JzZGF0YS5jbjtrbXMwMi10aHJvbmUwMS5zZW5zb3JzZGF0YS5jbjo5MjkyL2ttcw],
> exception [com.fasterxml.jackson.core.JsonParseException: Unexpected
> character ('<' (code 60)): expected a valid value (JSON String, Number,
> Array, Object or token 'null', 'true' or 'false') at [Source:
> (sun.net.www.protocol.http.HttpURLConnection$HttpInputStream); line: 1,
> column: 2]] at
> org.apache.hadoop.util.HttpExceptionUtils.validateResponse(HttpExceptionUtils.java:167)
> ~[classes/:?]at
> org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.doDelegationTokenOperation(DelegationTokenAuthenticator.java:318)
> ~[hadoop-common-3.3.4.jar:?] at
> org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.renewDelegationToken(DelegationTokenAuthenticator.java:235)
> ~[hadoop-common-3.3.4.jar:?]at
> org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.renewDelegationToken(DelegationTokenAuthenticatedURL.java:435)
> ~[hadoop-common-3.3.4.jar:?] at
> org.apache.hadoop.crypto.key.kms.KMSClientProvider$4.run(KMSClientProvider.java:1072)
> ~[hadoop-common-3.3.4.jar:?] at
> org.apache.hadoop.crypto.key.kms.KMSClientProvider$4.run(KMSClientProvider.java:1069)
> ~[hadoop-common-3.3.4.jar:?] at
> java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_332]
> at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_332]at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878)
> ~[hadoop-common-3.3.4.jar:?]at
>
[jira] [Commented] (RANGER-4327) Make ranger2.4 hdfs plugin compatible with hadoop 3.3.5 +
[ https://issues.apache.org/jira/browse/RANGER-4327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17805901#comment-17805901 ] Bhavik Patel commented on RANGER-4327: -- This issue is observed as jackson-1.x version is removed in the hadoop-3.4 : [https://github.com/apache/hadoop/commit/04b6b9a87bf024e82aa50328b4994aea4268d98e#diff-635a9ebcc70f58d8ca1c55b0d78cc90064a5081d189162a432271778da3cf018] >From ranger end also we have to remove jackson-1.x version as it's vulnerable. > Make ranger2.4 hdfs plugin compatible with hadoop 3.3.5 + > - > > Key: RANGER-4327 > URL: https://issues.apache.org/jira/browse/RANGER-4327 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: 2.3.0, 2.4.0 >Reporter: caijialiang >Assignee: caijialiang >Priority: Major > Attachments: image-2023-07-25-17-30-18-136.png, > image-2023-07-26-13-40-10-555.png, patch3-RANGER-4327.diff > > > hadoop 3.3.5 has upgraded its jackson dependency and removed all jackson 1.x > packages, including the jackson-jaxrs jar. As a result, when starting with > the Ranger plugin enabled, Hadoop namenode fails to start due to the > inability to load jackson 1.x. > To address this issue and ensure compatibility with the Hadoop 3.3.5 upgrade, > we need to package jackson 1.x jars in the hdfs plugin as well,like the Ozone > did https://issues.apache.org/jira/browse/RANGER-2818. This will allow the > namenode to start successfully. > !image-2023-07-26-13-40-10-555.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (RANGER-4505) [hdfs] Error - java.lang.ClassNotFoundException: org.codehaus.jackson.jaxrs.JacksonJsonProvider
[
https://issues.apache.org/jira/browse/RANGER-4505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bhavik Patel resolved RANGER-4505.
--
Resolution: Duplicate
> [hdfs] Error - java.lang.ClassNotFoundException:
> org.codehaus.jackson.jaxrs.JacksonJsonProvider
> ---
>
> Key: RANGER-4505
> URL: https://issues.apache.org/jira/browse/RANGER-4505
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Affects Versions: 2.3.0, 2.4.0
>Reporter: Anatoly
>Priority: Major
>
> ranger_version 2.3.0
> After upgrading the hadoop version from 3.1. to 3.3.6 and 3.3.5, I get an
> error for hdfs -
> java.lang.ClassNotFoundException:
> org.codehaus.jackson.jaxrs.JacksonJsonProvider
>
> {code:java}
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: 2023-10-31 14:55:05,181 ERROR
> namenode.NameNode: Failed to start namenode.
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: java.lang.NoClassDefFoundError:
> org/codehaus/jackson/jaxrs/JacksonJsonProvider
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.ranger.plugin.util.RangerRESTClient.buildClient(RangerRESTClient.java:226)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.ranger.plugin.util.RangerRESTClient.getClient(RangerRESTClient.java:193)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.ranger.plugin.util.RangerRESTClient.get(RangerRESTClient.java:473)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.ranger.admin.client.RangerAdminRESTClient.getRangerRolesDownloadResponse(RangerAdminRESTClient.java:1340)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.ranger.admin.client.RangerAdminRESTClient.getRolesIfUpdatedWithCred(RangerAdminRESTClient.java:1202)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.ranger.admin.client.RangerAdminRESTClient.getRolesIfUpdated(RangerAdminRESTClient.java:167)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.ranger.plugin.util.RangerRolesProvider.loadUserGroupRolesFromAdmin(RangerRolesProvider.java:183)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.ranger.plugin.util.RangerRolesProvider.loadUserGroupRoles(RangerRolesProvider.java:123)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.ranger.plugin.util.PolicyRefresher.loadRoles(PolicyRefresher.java:495)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.ranger.plugin.util.PolicyRefresher.startRefresher(PolicyRefresher.java:144)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:243)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.start(RangerHdfsAuthorizer.java:112)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.start(RangerHdfsAuthorizer.java:86)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startCommonServices(FSNamesystem.java:1321)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.hadoop.hdfs.server.namenode.NameNode.startCommonServices(NameNode.java:883)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:804)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:1033)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:1008)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1782)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1847)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: Caused by:
> java.lang.ClassNotFoundException:
> org.codehaus.jackson.jaxrs.JacksonJsonProvider
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> java.base/java.lang.ClassLoader.findClass(ClassLoader.java:718)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.ranger.plugin.classloader.RangerPluginClassLoader$MyClassLoader.findClass(RangerPluginClassLoader.java:316)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:588)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.ranger.plugin.classloader.RangerPluginClassLoader.loadClass(RangerPluginClassLoader.java:160)
> Oct 31 14:55:05 hadoop001
[jira] [Commented] (RANGER-4505) [hdfs] Error - java.lang.ClassNotFoundException: org.codehaus.jackson.jaxrs.JacksonJsonProvider
[
https://issues.apache.org/jira/browse/RANGER-4505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17805900#comment-17805900
]
Bhavik Patel commented on RANGER-4505:
--
This issue is observed as jackson-1.x version is removed in the hadoop-3.4 :
[https://github.com/apache/hadoop/commit/04b6b9a87bf024e82aa50328b4994aea4268d98e#diff-635a9ebcc70f58d8ca1c55b0d78cc90064a5081d189162a432271778da3cf018]
>From ranger end also we have to remove jackson-1.x version as it's vulnerable.
> [hdfs] Error - java.lang.ClassNotFoundException:
> org.codehaus.jackson.jaxrs.JacksonJsonProvider
> ---
>
> Key: RANGER-4505
> URL: https://issues.apache.org/jira/browse/RANGER-4505
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Affects Versions: 2.3.0, 2.4.0
>Reporter: Anatoly
>Priority: Major
>
> ranger_version 2.3.0
> After upgrading the hadoop version from 3.1. to 3.3.6 and 3.3.5, I get an
> error for hdfs -
> java.lang.ClassNotFoundException:
> org.codehaus.jackson.jaxrs.JacksonJsonProvider
>
> {code:java}
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: 2023-10-31 14:55:05,181 ERROR
> namenode.NameNode: Failed to start namenode.
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: java.lang.NoClassDefFoundError:
> org/codehaus/jackson/jaxrs/JacksonJsonProvider
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.ranger.plugin.util.RangerRESTClient.buildClient(RangerRESTClient.java:226)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.ranger.plugin.util.RangerRESTClient.getClient(RangerRESTClient.java:193)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.ranger.plugin.util.RangerRESTClient.get(RangerRESTClient.java:473)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.ranger.admin.client.RangerAdminRESTClient.getRangerRolesDownloadResponse(RangerAdminRESTClient.java:1340)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.ranger.admin.client.RangerAdminRESTClient.getRolesIfUpdatedWithCred(RangerAdminRESTClient.java:1202)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.ranger.admin.client.RangerAdminRESTClient.getRolesIfUpdated(RangerAdminRESTClient.java:167)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.ranger.plugin.util.RangerRolesProvider.loadUserGroupRolesFromAdmin(RangerRolesProvider.java:183)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.ranger.plugin.util.RangerRolesProvider.loadUserGroupRoles(RangerRolesProvider.java:123)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.ranger.plugin.util.PolicyRefresher.loadRoles(PolicyRefresher.java:495)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.ranger.plugin.util.PolicyRefresher.startRefresher(PolicyRefresher.java:144)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:243)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.start(RangerHdfsAuthorizer.java:112)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.start(RangerHdfsAuthorizer.java:86)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startCommonServices(FSNamesystem.java:1321)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.hadoop.hdfs.server.namenode.NameNode.startCommonServices(NameNode.java:883)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:804)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:1033)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:1008)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1782)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1847)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: Caused by:
> java.lang.ClassNotFoundException:
> org.codehaus.jackson.jaxrs.JacksonJsonProvider
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> java.base/java.lang.ClassLoader.findClass(ClassLoader.java:718)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
> org.apache.ranger.plugin.classloader.RangerPluginClassLoader$MyClassLoader.findClass(RangerPluginClassLoader.java:316)
> Oct 31 14:55:05 hadoop001 hdfs[1417563]: at
>
[jira] [Commented] (RANGER-4225) Possible Jackson serialization issue due to not comply with Java bean standards
[
https://issues.apache.org/jira/browse/RANGER-4225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17805899#comment-17805899
]
Bhavik Patel commented on RANGER-4225:
--
Can you completely remove the jackson-1.x dependency from Ranger?
> Possible Jackson serialization issue due to not comply with Java bean
> standards
> ---
>
> Key: RANGER-4225
> URL: https://issues.apache.org/jira/browse/RANGER-4225
> Project: Ranger
> Issue Type: Bug
> Components: Ranger
>Affects Versions: 2.4.0
>Reporter: Sercan Tekin
>Assignee: Sercan Tekin
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: RANGER-4225.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> +*PROBLEM:*+
> Transitive Jackson-2 dependencies are included into Ranger's classpath in my
> env and conflicted with Jackson-1 dependencies.
> Jackson-2 uses Javabean naming conventions to figure out the Json properties
> in a Java class, and some of the Ranger's model classes don't comply with the
> convention.
> For example, when the leading camelcase word is only one letter in length,
> then deserialized response is broken. The following is what I observed in
> Ranger v2.4.0;
> On Ranger UI side, this
> [code-block|https://github.com/apache/ranger/blob/ranger-1.2/security-admin/src/main/webapp/scripts/views/policies/PermissionList.js#L224-L229]
> attempts to read vXStrings key in map, but the corresponding response has
> vxstrings instead:
> {code:java}
> {
> "startIndex": 0,
> "pageSize": 200,
> "totalCount": 11,
> "resultSize": 11,
> "sortType": "asc",
> "sortBy": "id",
> "listSize": 11,
> "vxstrings": [< here! This has to be vXStrings
> {
> "value": "public",
> ... {code}
> And this difference causes below error while reading the property, therefore
> the corresponding dropdown has no values as excepted;
> {code:java}
> PermissionList.js?ver=build.version:226 Uncaught TypeError: Cannot read
> properties of undefined (reading 'map')
> at Object.results (PermissionList.js?ver=build.version:226:34)
> at Object.success (select2.js?ver=build.version:450:47)
> at u (jquery-3.3.1.min.js?ver=build.version:2:27457)
> at Object.fireWith [as resolveWith]
> (jquery-3.3.1.min.js?ver=build.version:2:28202)
> at k (jquery-3.3.1.min.js?ver=build.version:2:77651)
> at XMLHttpRequest.
> (jquery-3.3.1.min.js?ver=build.version:2:79907){code}
> +*REFERENCES:*+
> Please see this reference related to capital letters
> [http://futuretask.blogspot.com/2005/01/java-tip-6-dont-capitalize-first-two.html]
> "Don't capitalize first two letters of a bean property name. This is in our
> java standards. You should not create a java bean property name that begins
> with a capital letter in the 1st two places."
> Also you can see the same issue is reported here
> [https://stackoverflow.com/questions/30205006/why-does-jackson-2-not-recognize-the-first-capital-letter-if-the-leading-camel-c]
>
> +*SOLUTION:*+
> {{@JsonProperty}} annotation needs to be added for mapping the properties
> with their corresponding getter/setter methods. This will not effect Ranger's
> functionality directly, but it will provide consistency even if Jackson-2 is
> included into classpath.
> I have tested it locally after adding {{@JsonProperty}} and everything worked
> well.
> The PR is attached.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (RANGER-3409) Update Jackson and remove Codehaus version
[ https://issues.apache.org/jira/browse/RANGER-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17805898#comment-17805898 ] Bhavik Patel commented on RANGER-3409: -- Thanks [~sercan.tekin] but we have to fir the issue. As I can see in hadoop-3.4.0 they have completely remove the support for jackson-1.x : [https://github.com/apache/hadoop/commit/04b6b9a87bf024e82aa50328b4994aea4268d98e#diff-635a9ebcc70f58d8ca1c55b0d78cc90064a5081d189162a432271778da3cf018] > Update Jackson and remove Codehaus version > -- > > Key: RANGER-3409 > URL: https://issues.apache.org/jira/browse/RANGER-3409 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Affects Versions: 3.0.0 >Reporter: Andrew Charneski >Priority: Blocker > > An old version of Jackson (Codehaus Jackson 1.9.13) is still being used. > Jackson has since moved namespaces with a reorganized library structure. > Update all references to the older version to use the newer version (which is > currently used in some modules). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-3993) Upgrade okhttp dependency
[ https://issues.apache.org/jira/browse/RANGER-3993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797089#comment-17797089 ] Bhavik Patel commented on RANGER-3993: -- [~vikkumar] Will discuss the approach the here. Instead of defining dependencies in module's pom, use to override in parent pom and exclude from respective module's pom. For ex: for your case, exclude from admin pon and override using in parent pom? What's your thought on this? PS: Dropping my earlier review comment. Your suggestion also make sense but I thought of avoiding multiple exclusion changes so planning to go with changes at module level. And some dependency’s(version) are required for specific modules. Also, to control the dependency and their version I raised RANGER-4569. > Upgrade okhttp dependency > -- > > Key: RANGER-3993 > URL: https://issues.apache.org/jira/browse/RANGER-3993 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Bhavik Patel >Assignee: Bhavik Patel >Priority: Major > Attachments: 0001-RANGER-3993-Upgrade-okhttp-dependency.patch > > > Upgrade okhttp dependency -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-4590) Upgrade Tomcat version to 8.5.96
[ https://issues.apache.org/jira/browse/RANGER-4590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavik Patel updated RANGER-4590: - Attachment: 0001-RANGER-4590-Upgrade-Tomcat-version-to-8.5.96.patch > Upgrade Tomcat version to 8.5.96 > > > Key: RANGER-4590 > URL: https://issues.apache.org/jira/browse/RANGER-4590 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Affects Versions: 3.0.0 >Reporter: Bhavik Patel >Assignee: Bhavik Patel >Priority: Major > Attachments: 0001-RANGER-4590-Upgrade-Tomcat-version-to-8.5.96.patch > > > Upgrade Tomcat version to 8.5.96 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (RANGER-4602) Upgrade Tomcat to 8.5.96 in all Ranger services
[ https://issues.apache.org/jira/browse/RANGER-4602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavik Patel resolved RANGER-4602. -- Resolution: Duplicate > Upgrade Tomcat to 8.5.96 in all Ranger services > --- > > Key: RANGER-4602 > URL: https://issues.apache.org/jira/browse/RANGER-4602 > Project: Ranger > Issue Type: Task > Components: Ranger >Reporter: Sanket Shelar >Assignee: Sanket Shelar >Priority: Major > > Upgrade Tomcat to 8.5.96 in all Ranger services -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4602) Upgrade Tomcat to 8.5.96 in all Ranger services
[ https://issues.apache.org/jira/browse/RANGER-4602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796033#comment-17796033 ] Bhavik Patel commented on RANGER-4602: -- Hi [~sanketshelar] I already raised https://issues.apache.org/jira/browse/RANGER-4590 to upgrade tomcat version. I will raise review request in sometime. > Upgrade Tomcat to 8.5.96 in all Ranger services > --- > > Key: RANGER-4602 > URL: https://issues.apache.org/jira/browse/RANGER-4602 > Project: Ranger > Issue Type: Task > Components: Ranger >Reporter: Sanket Shelar >Assignee: Sanket Shelar >Priority: Major > > Upgrade Tomcat to 8.5.96 in all Ranger services -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4590) Upgrade Tomcat version to 8.5.96
Bhavik Patel created RANGER-4590: Summary: Upgrade Tomcat version to 8.5.96 Key: RANGER-4590 URL: https://issues.apache.org/jira/browse/RANGER-4590 Project: Ranger Issue Type: Improvement Components: Ranger Affects Versions: 3.0.0 Reporter: Bhavik Patel Assignee: Bhavik Patel Upgrade Tomcat version to 8.5.96 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-3823) Remove commons-dbcp and commons-pool dependency
[ https://issues.apache.org/jira/browse/RANGER-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavik Patel updated RANGER-3823: - Description: Remove commons-dbcp and commons-pool dependency as we are using Hikari-CP (was: Update commons-dbcp version to commons-dbcp2 – 2.9.0 commons-pool to commons-pool2 - 2.11.1) > Remove commons-dbcp and commons-pool dependency > > > Key: RANGER-3823 > URL: https://issues.apache.org/jira/browse/RANGER-3823 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Bhavik Patel >Priority: Major > > Remove commons-dbcp and commons-pool dependency as we are using Hikari-CP -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-3823) Remove commons-dbcp and commons-pool dependency
[ https://issues.apache.org/jira/browse/RANGER-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavik Patel updated RANGER-3823: - Summary: Remove commons-dbcp and commons-pool dependency (was: Update commons-dbcp2 and commons-pool2) > Remove commons-dbcp and commons-pool dependency > > > Key: RANGER-3823 > URL: https://issues.apache.org/jira/browse/RANGER-3823 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Bhavik Patel >Priority: Major > > Update commons-dbcp version to commons-dbcp2 – 2.9.0 > commons-pool to commons-pool2 - 2.11.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4583) Installation of Ranger plugin for YARN fails due to missing htrace core library
[
https://issues.apache.org/jira/browse/RANGER-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17794022#comment-17794022
]
Bhavik Patel commented on RANGER-4583:
--
With which version of hadoop you are trying to test? I haven’t observed the
same error.
> Installation of Ranger plugin for YARN fails due to missing htrace core
> library
> ---
>
> Key: RANGER-4583
> URL: https://issues.apache.org/jira/browse/RANGER-4583
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Affects Versions: 2.4.0
>Reporter: wangzhongwei
>Priority: Major
> Attachments: RANGER-4583.patch
>
>
> Installation of Ranger plugin for YARN fails with following error:
> {code:java}
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/htrace/core/Tracer$Builder
> at org.apache.hadoop.fs.FsTracer.get(FsTracer.java:42)
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3407)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:158)
> at
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3474)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3442)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:524)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:365)
> at
> org.apache.hadoop.security.alias.JavaKeyStoreProvider.initFileSystem(JavaKeyStoreProvider.java:89)
> at
> org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.(AbstractJavaKeyStoreProvider.java:85)
> at
> org.apache.hadoop.security.alias.JavaKeyStoreProvider.(JavaKeyStoreProvider.java:49)
> at
> org.apache.hadoop.security.alias.JavaKeyStoreProvider.(JavaKeyStoreProvider.java:41)
> at
> org.apache.hadoop.security.alias.JavaKeyStoreProvider$Factory.createProvider(JavaKeyStoreProvider.java:100)
> at
> org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:73)
> at
> org.apache.ranger.credentialapi.CredentialReader.getDecryptedString(CredentialReader.java:74)
> at
> org.apache.ranger.credentialapi.buildks.createCredential(buildks.java:87)
> at org.apache.ranger.credentialapi.buildks.main(buildks.java:41)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.htrace.core.Tracer$Builder
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> ... 16 more]
> Exiting plugin installation {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (RANGER-4581) Upgrade fasterxml jackson version to 2.16.0
Bhavik Patel created RANGER-4581: Summary: Upgrade fasterxml jackson version to 2.16.0 Key: RANGER-4581 URL: https://issues.apache.org/jira/browse/RANGER-4581 Project: Ranger Issue Type: Improvement Components: Ranger Affects Versions: 3.0.0 Reporter: Bhavik Patel Assignee: Bhavik Patel Upgrade fasterxml jackson version to 2.16.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-3823) Update commons-dbcp2 and commons-pool2
[ https://issues.apache.org/jira/browse/RANGER-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavik Patel updated RANGER-3823: - Attachment: image-2023-12-06-12-16-05-989.png > Update commons-dbcp2 and commons-pool2 > -- > > Key: RANGER-3823 > URL: https://issues.apache.org/jira/browse/RANGER-3823 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Bhavik Patel >Priority: Major > > Update commons-dbcp version to commons-dbcp2 – 2.9.0 > commons-pool to commons-pool2 - 2.11.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-3823) Update commons-dbcp2 and commons-pool2
[ https://issues.apache.org/jira/browse/RANGER-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavik Patel updated RANGER-3823: - Attachment: (was: image-2023-12-06-12-16-05-989.png) > Update commons-dbcp2 and commons-pool2 > -- > > Key: RANGER-3823 > URL: https://issues.apache.org/jira/browse/RANGER-3823 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Bhavik Patel >Priority: Major > > Update commons-dbcp version to commons-dbcp2 – 2.9.0 > commons-pool to commons-pool2 - 2.11.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4200) Upgrade jquery version to 3.6.1 in order to fix its own vulnerabilities
[ https://issues.apache.org/jira/browse/RANGER-4200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17793513#comment-17793513 ] Bhavik Patel commented on RANGER-4200: -- The latest released jquery version is 3.7.1. Can you update the patch? cc: [~Dhaval.Rajpara] / [~ni3galave] / [~mugdha.varadkar] > Upgrade jquery version to 3.6.1 in order to fix its own vulnerabilities > > > Key: RANGER-4200 > URL: https://issues.apache.org/jira/browse/RANGER-4200 > Project: Ranger > Issue Type: Bug > Components: admin >Reporter: chenyu >Assignee: chenyu >Priority: Major > Fix For: 3.0.0, 2.4.1 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4076) Support Java 17 for build and runtime
[ https://issues.apache.org/jira/browse/RANGER-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17793511#comment-17793511 ] Bhavik Patel commented on RANGER-4076: -- [~andrewluo] are you working this? > Support Java 17 for build and runtime > - > > Key: RANGER-4076 > URL: https://issues.apache.org/jira/browse/RANGER-4076 > Project: Ranger > Issue Type: New Feature > Components: admin, build-infra >Reporter: Andrew Luo >Priority: Major > Attachments: > 0001-RANGER-4076-Support-Java-17-for-build-and-runtime.patch > > > Currently only Java 8 and 11 are supported. Java 17 is a major LTS version > of Java and adding support would modernize our Java version support. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4404) Audit to hdfs for orc format feature stabilisation
[
https://issues.apache.org/jira/browse/RANGER-4404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17791532#comment-17791532
]
Bhavik Patel commented on RANGER-4404:
--
Hi [~rmani] Json to ORC conversion tool looks promising an if we processes
parallelly. Will check it out. Thanks. ===>Did you got chance to check on this?
> Audit to hdfs for orc format feature stabilisation
> --
>
> Key: RANGER-4404
> URL: https://issues.apache.org/jira/browse/RANGER-4404
> Project: Ranger
> Issue Type: Improvement
> Components: audit
>Affects Versions: 3.0.0, 2.4.0
>Reporter: Bhavik Patel
>Assignee: Bhavik Patel
>Priority: Major
>
> Currently if we have 50GB audit log file in spool directory then it is taking
> 4-5hr for the conversion and writing to HDFS.
> Also, we are observing below error logs
> {code:java}
> ERROR [AuditFileQueueSpool_hdfs_destWriter] provider.BaseAuditHandler: Error
> writing to log file.
> java.lang.RuntimeException: Overflow of newLength.
> smallBuffer.length=1073741824, nextElemLength=38
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.increaseBufferSpace(BytesColumnVector.java:311)
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:182)
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:207)
> at org.apache.ranger.audit.utils.ORCFileUtil.log(ORCFileUtil.java:143)
> at
> org.apache.ranger.audit.utils.RangerORCAuditWriter$1.run(RangerORCAuditWriter.java:77)
> at
> org.apache.ranger.audit.utils.RangerORCAuditWriter$1.run(RangerORCAuditWriter.java:73)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
> at
> org.apache.ranger.audit.provider.MiscUtil.executePrivilegedAction(MiscUtil.java:541)
> at
> org.apache.ranger.audit.utils.RangerORCAuditWriter.logAuditAsORC(RangerORCAuditWriter.java:73)
> at
> org.apache.ranger.audit.utils.RangerORCAuditWriter.logAsORC(RangerORCAuditWriter.java:159)
> at
> org.apache.ranger.audit.utils.RangerORCAuditWriter.log(RangerORCAuditWriter.java:112)
> at
> org.apache.ranger.audit.destination.HDFSAuditDestination.logJSON(HDFSAuditDestination.java:78)
> at
> org.apache.ranger.audit.destination.HDFSAuditDestination.log(HDFSAuditDestination.java:163)
> at
> org.apache.ranger.audit.queue.AuditFileQueueSpool.sendEvent(AuditFileQueueSpool.java:926)
> at
> org.apache.ranger.audit.queue.AuditFileQueueSpool.logEvent(AuditFileQueueSpool.java:913)
> at
> org.apache.ranger.audit.queue.AuditFileQueueSpool.runLogAudit(AuditFileQueueSpool.java:847)
> at
> org.apache.ranger.audit.queue.AuditFileQueueSpool.run(AuditFileQueueSpool.java:790)
> {code}
> hive-storage-api version upgrade(>=2.7.3) required to resolve the above error.
> Current version is 2.7.2
> cc: [~rmani] [~fateh288]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (RANGER-4570) Upgrade POI version to 5.2.5
[ https://issues.apache.org/jira/browse/RANGER-4570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavik Patel updated RANGER-4570: - Attachment: 0001-RANGER-4570-Upgrade-POI-version-to-5.2.5.patch > Upgrade POI version to 5.2.5 > > > Key: RANGER-4570 > URL: https://issues.apache.org/jira/browse/RANGER-4570 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Affects Versions: 3.0.0 >Reporter: Bhavik Patel >Assignee: Bhavik Patel >Priority: Major > Attachments: 0001-RANGER-4570-Upgrade-POI-version-to-5.2.5.patch > > > Upgrade POI version to 5.2.5 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4570) Upgrade POI version to 5.2.5
Bhavik Patel created RANGER-4570: Summary: Upgrade POI version to 5.2.5 Key: RANGER-4570 URL: https://issues.apache.org/jira/browse/RANGER-4570 Project: Ranger Issue Type: Improvement Components: Ranger Affects Versions: 3.0.0 Reporter: Bhavik Patel Assignee: Bhavik Patel Upgrade POI version to 5.2.5 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4569) Add "ranger_home/ews/webapp/WEB-INF/lib" path in the admin-web disto file
Bhavik Patel created RANGER-4569: Summary: Add "ranger_home/ews/webapp/WEB-INF/lib" path in the admin-web disto file Key: RANGER-4569 URL: https://issues.apache.org/jira/browse/RANGER-4569 Project: Ranger Issue Type: Improvement Components: Ranger Reporter: Bhavik Patel Currently there no specific module set is present in the admin-web.xml to control dependency’s in "ranger_home/ews/webapp/WEB-INF/lib". This will help to avoid unused jar from the packaging and tar ball size will also reduce. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-4568) Upgrade spring version to 5.3.31
[ https://issues.apache.org/jira/browse/RANGER-4568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavik Patel updated RANGER-4568: - Attachment: 0001-RANGER-4568-Upgrade-spring-version-to-5.3.31.patch > Upgrade spring version to 5.3.31 > > > Key: RANGER-4568 > URL: https://issues.apache.org/jira/browse/RANGER-4568 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Bhavik Patel >Assignee: Bhavik Patel >Priority: Major > Attachments: 0001-RANGER-4568-Upgrade-spring-version-to-5.3.31.patch > > > Upgrade spring version to 5.3.31 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-4568) Upgrade spring version to 5.3.31
[ https://issues.apache.org/jira/browse/RANGER-4568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavik Patel updated RANGER-4568: - Description: Upgrade spring version to 5.3.31 (was: Upgrade jetty-client to 9.4.53.v20231009) > Upgrade spring version to 5.3.31 > > > Key: RANGER-4568 > URL: https://issues.apache.org/jira/browse/RANGER-4568 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Bhavik Patel >Assignee: Bhavik Patel >Priority: Major > > Upgrade spring version to 5.3.31 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-4568) Upgrade spring version to 5.3.31
[ https://issues.apache.org/jira/browse/RANGER-4568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavik Patel updated RANGER-4568: - Summary: Upgrade spring version to 5.3.31 (was: Upgrade jetty-client to 9.4.53.v20231009) > Upgrade spring version to 5.3.31 > > > Key: RANGER-4568 > URL: https://issues.apache.org/jira/browse/RANGER-4568 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Bhavik Patel >Assignee: Bhavik Patel >Priority: Major > > Upgrade jetty-client to 9.4.53.v20231009 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4568) Upgrade jetty-client to 9.4.53.v20231009
Bhavik Patel created RANGER-4568: Summary: Upgrade jetty-client to 9.4.53.v20231009 Key: RANGER-4568 URL: https://issues.apache.org/jira/browse/RANGER-4568 Project: Ranger Issue Type: Improvement Components: Ranger Reporter: Bhavik Patel Assignee: Bhavik Patel Upgrade jetty-client to 9.4.53.v20231009 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-4567) Upgrade woodstox and stax2api version
[ https://issues.apache.org/jira/browse/RANGER-4567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavik Patel updated RANGER-4567: - Attachment: 0001-RANGER-4567-Upgrade-woodstox-and-stax2api-version.patch > Upgrade woodstox and stax2api version > - > > Key: RANGER-4567 > URL: https://issues.apache.org/jira/browse/RANGER-4567 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Affects Versions: 3.0.0 >Reporter: Bhavik Patel >Assignee: Bhavik Patel >Priority: Major > Attachments: > 0001-RANGER-4567-Upgrade-woodstox-and-stax2api-version.patch > > > Upgrade woodstox and stax2api version -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4567) Upgrade woodstox and stax2api version
Bhavik Patel created RANGER-4567: Summary: Upgrade woodstox and stax2api version Key: RANGER-4567 URL: https://issues.apache.org/jira/browse/RANGER-4567 Project: Ranger Issue Type: Improvement Components: Ranger Affects Versions: 3.0.0 Reporter: Bhavik Patel Assignee: Bhavik Patel Upgrade woodstox and stax2api version -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-3993) Upgrade okhttp dependency
[ https://issues.apache.org/jira/browse/RANGER-3993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790469#comment-17790469 ] Bhavik Patel commented on RANGER-3993: -- As this is the transitive dependency so we can fix this in two way: 1. upgrade hadoop version so with latest version will get the updated "okhttp" dependency 2. exclude from the hadoop dependency and include them externally. As you mentioned "can't we simply update in distro that defines the runtime required libs" ==>This will only help if we are downloading the dependency(whaterver the version specified in the distro), either direct or transitive dependency otherwise that jar will not be partof the packaging. > Upgrade okhttp dependency > -- > > Key: RANGER-3993 > URL: https://issues.apache.org/jira/browse/RANGER-3993 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Bhavik Patel >Assignee: Bhavik Patel >Priority: Major > Attachments: 0001-RANGER-3993-Upgrade-okhttp-dependency.patch > > > Upgrade okhttp dependency -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-3722) Fix Test-case failure
[ https://issues.apache.org/jira/browse/RANGER-3722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavik Patel updated RANGER-3722: - Fix Version/s: 3.0.0 > Fix Test-case failure > - > > Key: RANGER-3722 > URL: https://issues.apache.org/jira/browse/RANGER-3722 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 3.0.0 >Reporter: Bhavik Patel >Assignee: Bhavik Patel >Priority: Major > Fix For: 3.0.0 > > Attachments: 0001-RANGER-3722-Fix-Test-case-failure.patch > > > Below Test-case fail when we use maven surefire report plugins: > > test49importPoliciesFromFileAllowingOverride(org.apache.ranger.rest.TestServiceREST) > Time elapsed: 0.157 sec <<< ERROR! > java.io.FileNotFoundException: > ./src/test/java/org/apache/ranger/rest/importPolicy/import_policy_test_file.json > (No such file or directory) > at java.io.FileInputStream.open0(Native Method) > at java.io.FileInputStream.open(FileInputStream.java:195) > at java.io.FileInputStream.(FileInputStream.java:138) > at > org.apache.ranger.rest.TestServiceREST.test49importPoliciesFromFileAllowingOverride(TestServiceREST.java:1703) > > test50importPoliciesFromFileNotAllowingOverride(org.apache.ranger.rest.TestServiceREST) > Time elapsed: 0.003 sec <<< ERROR! > java.io.FileNotFoundException: > ./src/test/java/org/apache/ranger/rest/importPolicy/import_policy_test_file.json > (No such file or directory) > at java.io.FileInputStream.open0(Native Method) > at java.io.FileInputStream.open(FileInputStream.java:195) > at java.io.FileInputStream.(FileInputStream.java:138) > at > org.apache.ranger.rest.TestServiceREST.test50importPoliciesFromFileNotAllowingOverride(TestServiceREST.java:1768) > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4524) FIx TestRangerMetricsContainerSource class UT
[
https://issues.apache.org/jira/browse/RANGER-4524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17789919#comment-17789919
]
Bhavik Patel commented on RANGER-4524:
--
Commit link:
https://github.com/apache/ranger/commit/c7573d06726e42ced28f3e124095ab531c93b75b
> FIx TestRangerMetricsContainerSource class UT
> -
>
> Key: RANGER-4524
> URL: https://issues.apache.org/jira/browse/RANGER-4524
> Project: Ranger
> Issue Type: Improvement
> Components: Ranger
>Affects Versions: 3.0.0
>Reporter: Bhavik Patel
>Assignee: Bhavik Patel
>Priority: Major
> Fix For: 3.0.0
>
> Attachments:
> 0001-RANGER-4524-Fix-TestRangerMetricsContainerSource-cla.patch
>
>
> Steps to reproduce:
> 1. clean .m2 repo and trigger the test-cases.
> {code:java}
> java.lang.NoClassDefFoundError: org/apache/log4j/Level at
> org.apache.ranger.metrics.source.TestRangerMetricsContainerSource.init(TestRangerMetricsContainerSource.java:46)
> Caused by: java.lang.ClassNotFoundException: org.apache.log4j.Level at
> org.apache.ranger.metrics.source.TestRangerMetricsContainerSource.init(TestRangerMetricsContainerSource.java:46){code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (RANGER-4563) Upgrade Jetty-client version to 9.4.53.v20231009
[ https://issues.apache.org/jira/browse/RANGER-4563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavik Patel updated RANGER-4563: - Attachment: 0001-RANGER-4563-Upgrade-Jetty-client-version-to-9.4.53.v.patch > Upgrade Jetty-client version to 9.4.53.v20231009 > > > Key: RANGER-4563 > URL: https://issues.apache.org/jira/browse/RANGER-4563 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Affects Versions: 3.0.0 >Reporter: Bhavik Patel >Assignee: Bhavik Patel >Priority: Major > Attachments: > 0001-RANGER-4563-Upgrade-Jetty-client-version-to-9.4.53.v.patch > > > Upgrade Jetty-client version to 9.4.53.v20231009 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4563) Upgrade Jetty-client version to 9.4.53.v20231009
Bhavik Patel created RANGER-4563: Summary: Upgrade Jetty-client version to 9.4.53.v20231009 Key: RANGER-4563 URL: https://issues.apache.org/jira/browse/RANGER-4563 Project: Ranger Issue Type: Improvement Components: Ranger Affects Versions: 3.0.0 Reporter: Bhavik Patel Assignee: Bhavik Patel Upgrade Jetty-client version to 9.4.53.v20231009 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4441) Upgrade maven compiler plugin to 3.5
[ https://issues.apache.org/jira/browse/RANGER-4441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17789327#comment-17789327 ] Bhavik Patel commented on RANGER-4441: -- Can someone review the patch? > Upgrade maven compiler plugin to 3.5 > > > Key: RANGER-4441 > URL: https://issues.apache.org/jira/browse/RANGER-4441 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Affects Versions: 3.0.0, 2.4.0 >Reporter: Bhavik Patel >Assignee: Bhavik Patel >Priority: Major > Attachments: > 0001-RANGER-4441-Upgrade-maven-compiler-plugin-to-3.5.patch > > > Upgrade maven compiler plugin to 3.5 and current version 3.3 tried to > download log4j-1.2 jar. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (RANGER-3821) Update commons-codec version to 1.15
[ https://issues.apache.org/jira/browse/RANGER-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavik Patel reassigned RANGER-3821: Assignee: Bhavik Patel > Update commons-codec version to 1.15 > > > Key: RANGER-3821 > URL: https://issues.apache.org/jira/browse/RANGER-3821 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 3.0.0 >Reporter: Bhavik Patel >Assignee: Bhavik Patel >Priority: Major > Fix For: 3.0.0 > > Attachments: > 0001-RANGER-3821-Update-commons-codec-version-to-1.15.patch > > > update commons-codec version to 1.15 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4510) Ranger KMS - Upgrade Okhttp to 4.11.0 due to CVE-2023-0833 and CVE-2021-0341
[ https://issues.apache.org/jira/browse/RANGER-4510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788625#comment-17788625 ] Bhavik Patel commented on RANGER-4510: -- Hi [~vikkumar] , we already have [RANGER-3993 |https://reviews.apache.org/r/74228/bugs/RANGER-3993/]and I have raised the PR for the same. Can you review once. Thanks. > Ranger KMS - Upgrade Okhttp to 4.11.0 due to CVE-2023-0833 and CVE-2021-0341 > > > Key: RANGER-4510 > URL: https://issues.apache.org/jira/browse/RANGER-4510 > Project: Ranger > Issue Type: Task > Components: kms >Reporter: Vikas Kumar >Assignee: Vikas Kumar >Priority: Major > > Ranger KMS - Upgrade Okhttp to 4.11.0 due to CVE-2023-0833 and CVE-2021-0341 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (RANGER-4524) FIx TestRangerMetricsContainerSource class UT
[
https://issues.apache.org/jira/browse/RANGER-4524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786632#comment-17786632
]
Bhavik Patel edited comment on RANGER-4524 at 11/20/23 7:30 AM:
[~kishor.gollapalliwar] / [~dhavalshah9131] Can you review and merge?
was (Author: bpatel):
[~kishor.gollapalliwar] Can you review and merge?
> FIx TestRangerMetricsContainerSource class UT
> -
>
> Key: RANGER-4524
> URL: https://issues.apache.org/jira/browse/RANGER-4524
> Project: Ranger
> Issue Type: Improvement
> Components: Ranger
>Affects Versions: 3.0.0
>Reporter: Bhavik Patel
>Assignee: Bhavik Patel
>Priority: Major
> Fix For: 3.0.0
>
> Attachments:
> 0001-RANGER-4524-Fix-TestRangerMetricsContainerSource-cla.patch
>
>
> Steps to reproduce:
> 1. clean .m2 repo and trigger the test-cases.
> {code:java}
> java.lang.NoClassDefFoundError: org/apache/log4j/Level at
> org.apache.ranger.metrics.source.TestRangerMetricsContainerSource.init(TestRangerMetricsContainerSource.java:46)
> Caused by: java.lang.ClassNotFoundException: org.apache.log4j.Level at
> org.apache.ranger.metrics.source.TestRangerMetricsContainerSource.init(TestRangerMetricsContainerSource.java:46){code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (RANGER-4418) Upgrade hadoop version and use shaded hadoop client artifacts
[
https://issues.apache.org/jira/browse/RANGER-4418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787844#comment-17787844
]
Bhavik Patel commented on RANGER-4418:
--
Followed the build and installation cmd is mentioned in the readme file.
{code:java}
java.lang.NoClassDefFoundError:
org/apache/hadoop/thirdparty/com/google/common/collect/Interners {code}
> Upgrade hadoop version and use shaded hadoop client artifacts
> -
>
> Key: RANGER-4418
> URL: https://issues.apache.org/jira/browse/RANGER-4418
> Project: Ranger
> Issue Type: Improvement
> Components: Ranger
>Affects Versions: 2.3.0, 2.4.0
>Reporter: YUBI LEE
>Priority: Major
> Attachments: screenshot-1.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> - Upgrade the hadoop version Ranger uses.
> - Try to use shaded hadoop client artifacts if possible.
> Related issue: https://issues.apache.org/jira/browse/HADOOP-11804
> If shaded hadoop client artifacts ({{hadoop-client-api}},
> {{hadoop-client-runtime}}) are used, Ranger will be free to use any version
> of thirdparty libraries without collision.
> I will make a pull request soon.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (RANGER-3722) Fix Test-case failure
[ https://issues.apache.org/jira/browse/RANGER-3722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787127#comment-17787127 ] Bhavik Patel commented on RANGER-3722: -- Kindly review the changes. > Fix Test-case failure > - > > Key: RANGER-3722 > URL: https://issues.apache.org/jira/browse/RANGER-3722 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 3.0.0 >Reporter: Bhavik Patel >Assignee: Bhavik Patel >Priority: Major > Attachments: 0001-RANGER-3722-Fix-Test-case-failure.patch > > > Below Test-case fail when we use maven surefire report plugins: > > test49importPoliciesFromFileAllowingOverride(org.apache.ranger.rest.TestServiceREST) > Time elapsed: 0.157 sec <<< ERROR! > java.io.FileNotFoundException: > ./src/test/java/org/apache/ranger/rest/importPolicy/import_policy_test_file.json > (No such file or directory) > at java.io.FileInputStream.open0(Native Method) > at java.io.FileInputStream.open(FileInputStream.java:195) > at java.io.FileInputStream.(FileInputStream.java:138) > at > org.apache.ranger.rest.TestServiceREST.test49importPoliciesFromFileAllowingOverride(TestServiceREST.java:1703) > > test50importPoliciesFromFileNotAllowingOverride(org.apache.ranger.rest.TestServiceREST) > Time elapsed: 0.003 sec <<< ERROR! > java.io.FileNotFoundException: > ./src/test/java/org/apache/ranger/rest/importPolicy/import_policy_test_file.json > (No such file or directory) > at java.io.FileInputStream.open0(Native Method) > at java.io.FileInputStream.open(FileInputStream.java:195) > at java.io.FileInputStream.(FileInputStream.java:138) > at > org.apache.ranger.rest.TestServiceREST.test50importPoliciesFromFileNotAllowingOverride(TestServiceREST.java:1768) > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4524) FIx TestRangerMetricsContainerSource class UT
[
https://issues.apache.org/jira/browse/RANGER-4524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786632#comment-17786632
]
Bhavik Patel commented on RANGER-4524:
--
[~kishor.gollapalliwar] Can you review and merge?
> FIx TestRangerMetricsContainerSource class UT
> -
>
> Key: RANGER-4524
> URL: https://issues.apache.org/jira/browse/RANGER-4524
> Project: Ranger
> Issue Type: Improvement
> Components: Ranger
>Affects Versions: 3.0.0
>Reporter: Bhavik Patel
>Assignee: Bhavik Patel
>Priority: Major
> Fix For: 3.0.0
>
> Attachments:
> 0001-RANGER-4524-Fix-TestRangerMetricsContainerSource-cla.patch
>
>
> Steps to reproduce:
> 1. clean .m2 repo and trigger the test-cases.
> {code:java}
> java.lang.NoClassDefFoundError: org/apache/log4j/Level at
> org.apache.ranger.metrics.source.TestRangerMetricsContainerSource.init(TestRangerMetricsContainerSource.java:46)
> Caused by: java.lang.ClassNotFoundException: org.apache.log4j.Level at
> org.apache.ranger.metrics.source.TestRangerMetricsContainerSource.init(TestRangerMetricsContainerSource.java:46){code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (RANGER-4418) Upgrade hadoop version and use shaded hadoop client artifacts
[
https://issues.apache.org/jira/browse/RANGER-4418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786613#comment-17786613
]
Bhavik Patel commented on RANGER-4418:
--
If you find any further issue, please letting me know. ==> For me ranger admin
service installation it’s self failed while creating jceks files after fixing
that it again failed while starting the service.
> Upgrade hadoop version and use shaded hadoop client artifacts
> -
>
> Key: RANGER-4418
> URL: https://issues.apache.org/jira/browse/RANGER-4418
> Project: Ranger
> Issue Type: Improvement
> Components: Ranger
>Affects Versions: 2.3.0, 2.4.0
>Reporter: YUBI LEE
>Priority: Major
> Attachments: screenshot-1.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> - Upgrade the hadoop version Ranger uses.
> - Try to use shaded hadoop client artifacts if possible.
> Related issue: https://issues.apache.org/jira/browse/HADOOP-11804
> If shaded hadoop client artifacts ({{hadoop-client-api}},
> {{hadoop-client-runtime}}) are used, Ranger will be free to use any version
> of thirdparty libraries without collision.
> I will make a pull request soon.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (RANGER-4454) RangerKMS adds support for the SM4 encryption algorithm.
[ https://issues.apache.org/jira/browse/RANGER-4454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17784731#comment-17784731 ] Bhavik Patel commented on RANGER-4454: -- Can you confirm this change will not break existing keys functioning? > RangerKMS adds support for the SM4 encryption algorithm. > > > Key: RANGER-4454 > URL: https://issues.apache.org/jira/browse/RANGER-4454 > Project: Ranger > Issue Type: New Feature > Components: kms, Ranger >Affects Versions: 2.3.0 > Environment: !image-2023-10-04-08-31-03-261.png! >Reporter: xiaojunxiang >Priority: Major > Attachments: HDFS_SM4.jpg, Jira_HDFS_SM4.jpg, RANGER-4454-000.patch, > SM4_NotAvaliable.jpg > > Time Spent: 10m > Remaining Estimate: 0h > > SM4 is already supported in recent versions (3.4.0) of hdfs transparent > encryption, > So RangerKMS should adapt to this. > When I add in the region of the Encryption key used "SM4 / CTR/NoPadding" > algorithm, RangerKMS background will print "under Caused by: Java security. > NoSuchAlgorithmException: SM4 KeyGenerator not available" > > Hadoop website: > [https://apache.github.io/hadoop/hadoop-project-dist/hadoop-hdfs/TransparentEncryption.html] > Jira(HDFS supported SM4): https://issues.apache.org/jira/browse/HDFS-15098 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4237) Fix bugs in changepasswordutil.py and changeusernameutil.py
[ https://issues.apache.org/jira/browse/RANGER-4237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17784726#comment-17784726 ] Bhavik Patel commented on RANGER-4237: -- Ambari community is working on python3 support and PR is already raised couple of week back. > Fix bugs in changepasswordutil.py and changeusernameutil.py > --- > > Key: RANGER-4237 > URL: https://issues.apache.org/jira/browse/RANGER-4237 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.4.0 >Reporter: Abhishek Kumar >Assignee: Abhishek Kumar >Priority: Major > Fix For: 3.0.0 > > > Found bugs: > * changepasswordutil.py exposes passwords while taking input > * unused imports > * RMI call to java may hang indefinitely in case the command doesn't > complete, add a timeout. > * Redundant logging constructs > * Fix while loops for parsing user input > * Use argparse & getpass to parse input > * Avoid sys.exit(1) in function calls to be more reusable. > * migrate to python3 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-4525) Upgrade spotbugs plugin to 4.7.3.5
[ https://issues.apache.org/jira/browse/RANGER-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavik Patel updated RANGER-4525: - Attachment: 0001-RANGER-4525-Upgrade-spotbugs-plugin-to-4.7.3.5.patch > Upgrade spotbugs plugin to 4.7.3.5 > -- > > Key: RANGER-4525 > URL: https://issues.apache.org/jira/browse/RANGER-4525 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Affects Versions: 3.0.0, 2.4.0 >Reporter: Bhavik Patel >Assignee: Bhavik Patel >Priority: Major > Attachments: 0001-RANGER-4525-Upgrade-spotbugs-plugin-to-4.7.3.5.patch > > > Upgrade spotbugs plugin to 4.7.3.5 as current version download the > commons-text-1.9 dependency and which is vulnerable. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4525) Upgrade spotbugs plugin to 4.7.3.5
Bhavik Patel created RANGER-4525: Summary: Upgrade spotbugs plugin to 4.7.3.5 Key: RANGER-4525 URL: https://issues.apache.org/jira/browse/RANGER-4525 Project: Ranger Issue Type: Improvement Components: Ranger Affects Versions: 2.4.0, 3.0.0 Reporter: Bhavik Patel Assignee: Bhavik Patel Upgrade spotbugs plugin to 4.7.3.5 as current version download the commons-text-1.9 dependency and which is vulnerable. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-4441) Upgrade maven compiler plugin to 3.5
[ https://issues.apache.org/jira/browse/RANGER-4441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavik Patel updated RANGER-4441: - Attachment: 0001-RANGER-4441-Upgrade-maven-compiler-plugin-to-3.5.patch > Upgrade maven compiler plugin to 3.5 > > > Key: RANGER-4441 > URL: https://issues.apache.org/jira/browse/RANGER-4441 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Affects Versions: 3.0.0, 2.4.0 >Reporter: Bhavik Patel >Assignee: Bhavik Patel >Priority: Major > Attachments: > 0001-RANGER-4441-Upgrade-maven-compiler-plugin-to-3.5.patch > > > Upgrade maven compiler plugin to 3.5 and current version 3.3 tried to > download log4j-1.2 jar. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-4524) FIx TestRangerMetricsContainerSource class UT
[
https://issues.apache.org/jira/browse/RANGER-4524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bhavik Patel updated RANGER-4524:
-
Description:
Steps to reproduce:
1. clean .m2 repo and trigger the test-cases.
{code:java}
java.lang.NoClassDefFoundError: org/apache/log4j/Level at
org.apache.ranger.metrics.source.TestRangerMetricsContainerSource.init(TestRangerMetricsContainerSource.java:46)
Caused by: java.lang.ClassNotFoundException: org.apache.log4j.Level at
org.apache.ranger.metrics.source.TestRangerMetricsContainerSource.init(TestRangerMetricsContainerSource.java:46){code}
was:java.lang.NoClassDefFoundError: org/apache/log4j/Level at
org.apache.ranger.metrics.source.TestRangerMetricsContainerSource.init(TestRangerMetricsContainerSource.java:46)
Caused by: java.lang.ClassNotFoundException: org.apache.log4j.Level at
org.apache.ranger.metrics.source.TestRangerMetricsContainerSource.init(TestRangerMetricsContainerSource.java:46)
> FIx TestRangerMetricsContainerSource class UT
> -
>
> Key: RANGER-4524
> URL: https://issues.apache.org/jira/browse/RANGER-4524
> Project: Ranger
> Issue Type: Improvement
> Components: Ranger
>Affects Versions: 3.0.0
>Reporter: Bhavik Patel
>Assignee: Bhavik Patel
>Priority: Major
> Fix For: 3.0.0
>
> Attachments:
> 0001-RANGER-4524-Fix-TestRangerMetricsContainerSource-cla.patch
>
>
> Steps to reproduce:
> 1. clean .m2 repo and trigger the test-cases.
> {code:java}
> java.lang.NoClassDefFoundError: org/apache/log4j/Level at
> org.apache.ranger.metrics.source.TestRangerMetricsContainerSource.init(TestRangerMetricsContainerSource.java:46)
> Caused by: java.lang.ClassNotFoundException: org.apache.log4j.Level at
> org.apache.ranger.metrics.source.TestRangerMetricsContainerSource.init(TestRangerMetricsContainerSource.java:46){code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (RANGER-4524) FIx TestRangerMetricsContainerSource class UT
[ https://issues.apache.org/jira/browse/RANGER-4524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavik Patel updated RANGER-4524: - Attachment: 0001-RANGER-4524-Fix-TestRangerMetricsContainerSource-cla.patch > FIx TestRangerMetricsContainerSource class UT > - > > Key: RANGER-4524 > URL: https://issues.apache.org/jira/browse/RANGER-4524 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Affects Versions: 3.0.0 >Reporter: Bhavik Patel >Assignee: Bhavik Patel >Priority: Major > Fix For: 3.0.0 > > Attachments: > 0001-RANGER-4524-Fix-TestRangerMetricsContainerSource-cla.patch > > > java.lang.NoClassDefFoundError: org/apache/log4j/Level at > org.apache.ranger.metrics.source.TestRangerMetricsContainerSource.init(TestRangerMetricsContainerSource.java:46) > Caused by: java.lang.ClassNotFoundException: org.apache.log4j.Level at > org.apache.ranger.metrics.source.TestRangerMetricsContainerSource.init(TestRangerMetricsContainerSource.java:46) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4524) FIx TestRangerMetricsContainerSource class UT
Bhavik Patel created RANGER-4524: Summary: FIx TestRangerMetricsContainerSource class UT Key: RANGER-4524 URL: https://issues.apache.org/jira/browse/RANGER-4524 Project: Ranger Issue Type: Improvement Components: Ranger Affects Versions: 3.0.0 Reporter: Bhavik Patel Assignee: Bhavik Patel Fix For: 3.0.0 java.lang.NoClassDefFoundError: org/apache/log4j/Level at org.apache.ranger.metrics.source.TestRangerMetricsContainerSource.init(TestRangerMetricsContainerSource.java:46) Caused by: java.lang.ClassNotFoundException: org.apache.log4j.Level at org.apache.ranger.metrics.source.TestRangerMetricsContainerSource.init(TestRangerMetricsContainerSource.java:46) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4418) Upgrade hadoop version and use shaded hadoop client artifacts
[
https://issues.apache.org/jira/browse/RANGER-4418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1190#comment-1190
]
Bhavik Patel commented on RANGER-4418:
--
how did you validated your changes?
> Upgrade hadoop version and use shaded hadoop client artifacts
> -
>
> Key: RANGER-4418
> URL: https://issues.apache.org/jira/browse/RANGER-4418
> Project: Ranger
> Issue Type: Improvement
> Components: Ranger
>Affects Versions: 2.3.0, 2.4.0
>Reporter: YUBI LEE
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> - Upgrade the hadoop version Ranger uses.
> - Try to use shaded hadoop client artifacts if possible.
> Related issue: https://issues.apache.org/jira/browse/HADOOP-11804
> If shaded hadoop client artifacts ({{hadoop-client-api}},
> {{hadoop-client-runtime}}) are used, Ranger will be free to use any version
> of thirdparty libraries without collision.
> I will make a pull request soon.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (RANGER-4401) Configurable Graalvm features
[ https://issues.apache.org/jira/browse/RANGER-4401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1018#comment-1018 ] Bhavik Patel commented on RANGER-4401: -- [~kishor.gollapalliwar] do we have any wiki guidelines to configure the required properties ? or you can list down in the Jira. Also, we have to uncomment this part of code, right? https://github.com/apache/ranger/blob/master/agents-common/pom.xml#L167 > Configurable Graalvm features > - > > Key: RANGER-4401 > URL: https://issues.apache.org/jira/browse/RANGER-4401 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Kishor Gollapalliwar >Assignee: Kishor Gollapalliwar >Priority: Major > Fix For: 3.0.0 > > > Currently the only way of enabling GraalVm features/ options is by passing > JVM options. Which might not be feasible always. Hence we need a plugin > config, which will make GraalVm feature enabling configurable. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4476) trino filter
[
https://issues.apache.org/jira/browse/RANGER-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776673#comment-17776673
]
Bhavik Patel commented on RANGER-4476:
--
With java-8 and 11 we are able to build without any code changes but with
jdk-17 I am observing the above issue.
> trino filter
>
>
> Key: RANGER-4476
> URL: https://issues.apache.org/jira/browse/RANGER-4476
> Project: Ranger
> Issue Type: Bug
> Components: plugins, Ranger
>Affects Versions: 3.0.0
>Reporter: Andrey Pilipyuk
>Priority: Major
> Labels: Trino
>
> hello there
> i trying to enable trino plugin, i have successful connect check
> but when i try to make policy i have an error in logs
> 2023-10-16 13:39:09,055 [https-jsse-nio-6182-exec-6] WARN
> [ServiceMgr.java:476] getFilesInDirectory('ranger-plugins/trino'): adding
> /opt/ranger-admin-3.0.0/ews/webapp/WEB-INF/classes/ranger-plugins/trino/trino-jdbc-423.jar
> 2023-10-16 13:39:09,055 [https-jsse-nio-6182-exec-6] WARN
> [ServiceMgr.java:476] getFilesInDirectory('ranger-plugins/trino'): adding
> /opt/ranger-admin-3.0.0/ews/webapp/WEB-INF/classes/ranger-plugins/trino/ranger-trino-plugin-3.0.0-SNAPSHOT.jar
> 2023-10-16 13:39:09,056 [https-jsse-nio-6182-exec-6] WARN
> [ServiceMgr.java:476] getFilesInDirectory('ranger-plugins/trino'): adding
> /opt/ranger-admin-3.0.0/ews/webapp/WEB-INF/classes/ranger-plugins/trino/trino-spi-423.jar
> 2023-10-16 13:39:09,078 [timed-executor-pool-0] WARN
> [SecureClientLogin.java:123] Can't find keyTab Path : null
> 2023-10-16 13:39:09,079 [timed-executor-pool-0] WARN
> [SecureClientLogin.java:127] Can't find principal : null
> 2023-10-16 13:39:09,084 [timed-executor-pool-0] INFO [BaseClient.java:126]
> Init Login: security not enabled, using username
> 2023-10-16 13:39:09,501 [timed-executor-pool-0] ERROR
> [TrinoResourceManager.java:168] Could not initiate a TrinoClient timedTask
> can you please tell me how i can debug it, or some prefirences to trino from
> ranger
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (RANGER-4476) trino filter
[
https://issues.apache.org/jira/browse/RANGER-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776602#comment-17776602
]
Bhavik Patel commented on RANGER-4476:
--
* Trino latest version only support jdk-17 only.
* With jdk-17 I am observing the below error while build the trino plugin
{code:java}
agents-common/src/main/java/org/apache/ranger/plugin/util/NashornScriptEngineCreator.java:[26,32]
error: package jdk.nashorn.api.scripting does not exist{code}
how did resolved this?
> trino filter
>
>
> Key: RANGER-4476
> URL: https://issues.apache.org/jira/browse/RANGER-4476
> Project: Ranger
> Issue Type: Bug
> Components: plugins, Ranger
>Affects Versions: 3.0.0
>Reporter: Andrey Pilipyuk
>Priority: Major
> Labels: Trino
>
> hello there
> i trying to enable trino plugin, i have successful connect check
> but when i try to make policy i have an error in logs
> 2023-10-16 13:39:09,055 [https-jsse-nio-6182-exec-6] WARN
> [ServiceMgr.java:476] getFilesInDirectory('ranger-plugins/trino'): adding
> /opt/ranger-admin-3.0.0/ews/webapp/WEB-INF/classes/ranger-plugins/trino/trino-jdbc-423.jar
> 2023-10-16 13:39:09,055 [https-jsse-nio-6182-exec-6] WARN
> [ServiceMgr.java:476] getFilesInDirectory('ranger-plugins/trino'): adding
> /opt/ranger-admin-3.0.0/ews/webapp/WEB-INF/classes/ranger-plugins/trino/ranger-trino-plugin-3.0.0-SNAPSHOT.jar
> 2023-10-16 13:39:09,056 [https-jsse-nio-6182-exec-6] WARN
> [ServiceMgr.java:476] getFilesInDirectory('ranger-plugins/trino'): adding
> /opt/ranger-admin-3.0.0/ews/webapp/WEB-INF/classes/ranger-plugins/trino/trino-spi-423.jar
> 2023-10-16 13:39:09,078 [timed-executor-pool-0] WARN
> [SecureClientLogin.java:123] Can't find keyTab Path : null
> 2023-10-16 13:39:09,079 [timed-executor-pool-0] WARN
> [SecureClientLogin.java:127] Can't find principal : null
> 2023-10-16 13:39:09,084 [timed-executor-pool-0] INFO [BaseClient.java:126]
> Init Login: security not enabled, using username
> 2023-10-16 13:39:09,501 [timed-executor-pool-0] ERROR
> [TrinoResourceManager.java:168] Could not initiate a TrinoClient timedTask
> can you please tell me how i can debug it, or some prefirences to trino from
> ranger
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (RANGER-4476) trino filter
[
https://issues.apache.org/jira/browse/RANGER-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776561#comment-17776561
]
Bhavik Patel commented on RANGER-4476:
--
which version of trino you are using? and how you compiled the trino plugin?
> trino filter
>
>
> Key: RANGER-4476
> URL: https://issues.apache.org/jira/browse/RANGER-4476
> Project: Ranger
> Issue Type: Bug
> Components: plugins, Ranger
>Affects Versions: 3.0.0
>Reporter: Andrey Pilipyuk
>Priority: Major
> Labels: Trino
>
> hello there
> i trying to enable trino plugin, i have successful connect check
> but when i try to make policy i have an error in logs
> 2023-10-16 13:39:09,055 [https-jsse-nio-6182-exec-6] WARN
> [ServiceMgr.java:476] getFilesInDirectory('ranger-plugins/trino'): adding
> /opt/ranger-admin-3.0.0/ews/webapp/WEB-INF/classes/ranger-plugins/trino/trino-jdbc-423.jar
> 2023-10-16 13:39:09,055 [https-jsse-nio-6182-exec-6] WARN
> [ServiceMgr.java:476] getFilesInDirectory('ranger-plugins/trino'): adding
> /opt/ranger-admin-3.0.0/ews/webapp/WEB-INF/classes/ranger-plugins/trino/ranger-trino-plugin-3.0.0-SNAPSHOT.jar
> 2023-10-16 13:39:09,056 [https-jsse-nio-6182-exec-6] WARN
> [ServiceMgr.java:476] getFilesInDirectory('ranger-plugins/trino'): adding
> /opt/ranger-admin-3.0.0/ews/webapp/WEB-INF/classes/ranger-plugins/trino/trino-spi-423.jar
> 2023-10-16 13:39:09,078 [timed-executor-pool-0] WARN
> [SecureClientLogin.java:123] Can't find keyTab Path : null
> 2023-10-16 13:39:09,079 [timed-executor-pool-0] WARN
> [SecureClientLogin.java:127] Can't find principal : null
> 2023-10-16 13:39:09,084 [timed-executor-pool-0] INFO [BaseClient.java:126]
> Init Login: security not enabled, using username
> 2023-10-16 13:39:09,501 [timed-executor-pool-0] ERROR
> [TrinoResourceManager.java:168] Could not initiate a TrinoClient timedTask
> can you please tell me how i can debug it, or some prefirences to trino from
> ranger
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (RANGER-4239) Upgrade trino version
[ https://issues.apache.org/jira/browse/RANGER-4239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776474#comment-17776474 ] Bhavik Patel commented on RANGER-4239: -- [~ErikAnderson] you can create the separate Jira to upgrade the Trino version in ranger and provide the patch with validation. Will review and get it merge. Additionally, this community is nearly unresponsive to the speed of changes that would needed to maintain such a plugin inside Apache Ranger itself. ===> Let's discuss this point over separate mail thread rather than discussing over Jira which is assigned to someone else. > Upgrade trino version > - > > Key: RANGER-4239 > URL: https://issues.apache.org/jira/browse/RANGER-4239 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Affects Versions: 3.0.0 >Reporter: qingbo jiao >Assignee: Siddhant Sontakke >Priority: Minor > Time Spent: 0.5h > Remaining Estimate: 0h > > The current version of trino integrated in the ranger is 377, and the latest > version of trino has reached 417. Should the version of trino be upgraded to > 4XX? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4239) Upgrade trino version
[ https://issues.apache.org/jira/browse/RANGER-4239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776134#comment-17776134 ] Bhavik Patel commented on RANGER-4239: -- [~siddhant_sontakke] the review link is for RANGER-4203. Can you kindly update the attach the correct link? > Upgrade trino version > - > > Key: RANGER-4239 > URL: https://issues.apache.org/jira/browse/RANGER-4239 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Affects Versions: 3.0.0 >Reporter: qingbo jiao >Assignee: Siddhant Sontakke >Priority: Minor > Time Spent: 0.5h > Remaining Estimate: 0h > > The current version of trino integrated in the ranger is 377, and the latest > version of trino has reached 417. Should the version of trino be upgraded to > 4XX? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-3850) Upgrade trino plugin to JDK17
[ https://issues.apache.org/jira/browse/RANGER-3850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773999#comment-17773999 ] Bhavik Patel commented on RANGER-3850: -- [~aakashnand] Thanks for working on this and kindly handle the review comment. Is `getColumnMask` and `getRowFilter` supported in trino-417 version ? As in https://issues.apache.org/jira/browse/RANGER-3983 it is mentioned like support removed since 401 version. Can you confirm on this? > Upgrade trino plugin to JDK17 > - > > Key: RANGER-3850 > URL: https://issues.apache.org/jira/browse/RANGER-3850 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: 2.3.0 >Reporter: Aakash Nand >Priority: Major > Fix For: 3.0.0 > > Attachments: 0001-RANGER-3850-Upgrade-trino-plugin-to-JDK17.patch > > > From version 390 onwards, trino have started using JDK17 and from trino > version 393 onwards the language level will also be updated to JDK17. > Therefore trino plugin needs to be updated -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4441) Upgrade maven compiler plugin to 3.5
Bhavik Patel created RANGER-4441: Summary: Upgrade maven compiler plugin to 3.5 Key: RANGER-4441 URL: https://issues.apache.org/jira/browse/RANGER-4441 Project: Ranger Issue Type: Improvement Components: Ranger Affects Versions: 2.4.0, 3.0.0 Reporter: Bhavik Patel Assignee: Bhavik Patel Upgrade maven compiler plugin to 3.5 and current version 3.3 tried to download log4j-1.2 jar. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4418) Upgrade hadoop version and use shaded hadoop client artifacts
[
https://issues.apache.org/jira/browse/RANGER-4418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768603#comment-17768603
]
Bhavik Patel commented on RANGER-4418:
--
[~eub] Thanks for working this.
Any specific reason to use hadoop-3.3.4? Can you upgrade hadoop-3.3.6 as it
looks like more stable.
> Upgrade hadoop version and use shaded hadoop client artifacts
> -
>
> Key: RANGER-4418
> URL: https://issues.apache.org/jira/browse/RANGER-4418
> Project: Ranger
> Issue Type: Improvement
> Components: Ranger
>Affects Versions: 2.3.0, 2.4.0
>Reporter: YUBI LEE
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> - Upgrade the hadoop version Ranger uses.
> - Try to use shaded hadoop client artifacts if possible.
> Related issue: https://issues.apache.org/jira/browse/HADOOP-11804
> If shaded hadoop client artifacts ({{hadoop-client-api}},
> {{hadoop-client-runtime}}) are used, Ranger will be free to use any version
> of thirdparty libraries without collision.
> I will make a pull request soon.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (RANGER-4022) Facing Ranger AD sync issue
[
https://issues.apache.org/jira/browse/RANGER-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767509#comment-17767509
]
Bhavik Patel commented on RANGER-4022:
--
Is this resolved? If no can you check "rangerusersync" user is present in the
Ranger Admin or not?
> Facing Ranger AD sync issue
> ---
>
> Key: RANGER-4022
> URL: https://issues.apache.org/jira/browse/RANGER-4022
> Project: Ranger
> Issue Type: Task
> Components: usersync
>Affects Versions: 2.0.0
>Reporter: Ajay
>Priority: Major
>
> Hi Team,
>
> I am working on creating Open_source KAFKA/RANGER/AMBARI cluster , however
> everything has been setup but facing error while RANGER_AD sync. So Ranger
> admin and Ranger usersync are getting started via Ambari however with below
> errors it is getting failed. I am at a point where i am not able to find
> where the issue is at , any help will be appreciate able.
>
> Below is the error snap.
>
> Note:- this is a sample user taken from Ldap
> {code:java}
> 13 Dec 2022 18:19:42 INFO UnixAuthenticationService [main] - Starting User
> Sync Service!
> 13 Dec 2022 18:19:43 INFO AbstractMapper [UnixUserSyncThread] - Initializing
> for ranger.usersync.mapping.username.regex
> 13 Dec 2022 18:19:43 INFO AbstractMapper [UnixUserSyncThread] - Initializing
> for ranger.usersync.mapping.groupname.regex
> 13 Dec 2022 18:19:43 INFO LdapDeltaUserGroupBuilder [UnixUserSyncThread] -
> LdapDeltaUserGroupBuilder created
> 13 Dec 2022 18:19:43 INFO UserGroupSyncConfig [UnixUserSyncThread] - Sleep
> Time Between Cycle can not be lower than [360] millisec. resetting to min
> value.
> 13 Dec 2022 18:19:43 INFO UserGroupSync [UnixUserSyncThread] - initializing
> sink: org.apache.ranger.ldapusersync.process.LdapPolicyMgrUserGroupBuilder
> 13 Dec 2022 18:19:44 DEBUG Tracer [UnixUserSyncThread] - sampler.classes = ;
> loaded no samplers
> 13 Dec 2022 18:19:44 DEBUG Tracer [UnixUserSyncThread] -
> span.receiver.classes = ; loaded no span receivers
> 13 Dec 2022 18:19:45 INFO AbstractMapper [UnixUserSyncThread] - Initializing
> for ranger.usersync.mapping.username.regex
> 13 Dec 2022 18:19:45 INFO AbstractMapper [UnixUserSyncThread] - Initializing
> for ranger.usersync.mapping.groupname.regex
> 13 Dec 2022 18:19:45 INFO LdapDeltaUserGroupBuilder [UnixUserSyncThread] -
> LdapDeltaUserGroupBuilder created
> 13 Dec 2022 18:19:45 INFO UserGroupSync [UnixUserSyncThread] - initializing
> source: org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder
> 13 Dec 2022 18:19:45 INFO LdapDeltaUserGroupBuilder [UnixUserSyncThread] -
> LdapDeltaUserGroupBuilder initialization started
> 13 Dec 2022 18:19:46 INFO LdapDeltaUserGroupBuilder [UnixUserSyncThread] -
> LdapDeltaUserGroupBuilder initialization completed with -- ldapUrl:
> ldap://ldap-aws-us-east.mstarext.com:389, ldapBindDn:
> CN=aws_hadoop_prd_ad_user,OU=Service Accounts,OU=Hadoop,OU=CORESVC_Core
> Services,OU=Servers and Services,DC=mstarext,DC=com, ldapBindPassword: *
> , ldapAuthenticationMechanism: simple, searchBase: DC=mstarext,DC=com,
> userSearchBase: [dc=mstarext,dc=com], userSearchScope: 2, userObjectClass:
> user, userSearchFilter: (&(objectClass=person)(objectClass=user)),
> extendedUserSearchFilter: null, userNameAttribute: sAMAccountName,
> userSearchAttributes: [uSNChanged, sAMAccountName, modifytimestamp],
> userGroupNameAttributeSet: null, pagedResultsEnabled: true,
> pagedResultsSize: 500, groupSearchEnabled: true, groupSearchBase:
> [DC=mstarext,DC=com], groupSearchScope: 2, groupObjectClass: group,
> groupSearchFilter: (objectClass=group), extendedGroupSearchFilter:
> ((|(member={0})(member={1}))), extendedAllGroupsSearchFilter: null,
> groupMemberAttributeName: member, groupNameAttribute: sAMAccountName,
> groupSearchAttributes: [uSNChanged, sAMAccountName, member, modifytimestamp],
> groupUserMapSyncEnabled: true, groupSearchFirstEnabled: false,
> userSearchEnabled: false, ldapReferral: follow
> 13 Dec 2022 18:19:46 INFO UserGroupSync [UnixUserSyncThread] - Begin:
> initial load of user/group from source==>sink
> 13 Dec 2022 18:19:46 INFO LdapDeltaUserGroupBuilder [UnixUserSyncThread] -
> LdapDeltaUserGroupBuilder updateSink started
> 13 Dec 2022 18:19:46 INFO LdapDeltaUserGroupBuilder [UnixUserSyncThread] -
> Performing user search first
> 13 Dec 2022 18:19:46 INFO LdapDeltaUserGroupBuilder [UnixUserSyncThread] -
> extendedUserSearchFilter =
> (&(objectclass=user)(|(uSNChanged>=0)(modifyTimestamp>=1970010112Z))(&(objectClass=person)(objectClass=user)))
> 13 Dec 2022 18:19:46 INFO LdapDeltaUserGroupBuilder [UnixUserSyncThread] -
> uSNChangedVal = 77639505and currentDeltaSyncTime = 77639505
> 13 Dec 2022 18:19:46 DEBUG LdapPolicyMgrUserGroupBuilder [UnixUserSyncThread]
>
[jira] [Comment Edited] (RANGER-4404) Audit to hdfs for orc format feature stabilisation
[
https://issues.apache.org/jira/browse/RANGER-4404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764818#comment-17764818
]
Bhavik Patel edited comment on RANGER-4404 at 9/13/23 6:26 PM:
---
Observed these 4 for loop so first thought of checking whether we can avoind
any or not?
{code:java}
1 for loop in AuditFileQueueSpool(method logEvent(BufferedReader br)) =>
converting line to AuditEventBase Object
1 for loop in HDFSAuditDestination(method log(Collection
events)) => converting event to List of String
1 for loop in RangerORCAuditWriter(method
getAuthzAuditEvents(Collection events)) => converting event to
AuthzAuditEvent Object
1 for loop in ORCFileUtil (method log(Writer writer,
Collection events) => Actul mapping/adding the event data to
ORC{code}
So to avoid all the 2 for loop I updated the main method in the ORCFileUtil
class and tried to convert 10GB file so after 25min I got OOM(I explicitly set
heap size to 20GB).
{color:#de350b}Note: Also, I directly tried to use apache ORC-tools to convert
4.7GB json .log file to .ORC and only it took only 3min. {color}
So now thinking to check some-other approach :: about writing the logs directly
in .orc format in spool directory and then move to HDFS?
was (Author: bpatel):
Observed these 4 for loop so first thought of checking whether we can avoind
any or not?
{code:java}
1 for loop in AuditFileQueueSpool(method logEvent(BufferedReader br)) =>
converting line to AuditEventBase Object
1 for loop in HDFSAuditDestination(method log(Collection
events)) => converting event to List of String
1 for loop in RangerORCAuditWriter(method
getAuthzAuditEvents(Collection events)) => converting event to
AuthzAuditEvent Object
1 for loop in ORCFileUtil (method log(Writer writer,
Collection events) => Actul mapping/adding the event data to
ORC{code}
So to avoid all the 2 for loop I updated the main method in the ORCFileUtil
class and tried to convert 10GB file so after 25min I got OOM(I explicitly set
heap size to 20GB).
So now thinking to check some-other approach.
{color:#de350b}Note: Also, I directly tried to use apache ORC-tools to convert
4.7GB json .log file to .ORC and only it took only 3min.
{color}
> Audit to hdfs for orc format feature stabilisation
> --
>
> Key: RANGER-4404
> URL: https://issues.apache.org/jira/browse/RANGER-4404
> Project: Ranger
> Issue Type: Improvement
> Components: audit
>Affects Versions: 3.0.0, 2.4.0
>Reporter: Bhavik Patel
>Assignee: Bhavik Patel
>Priority: Major
>
> Currently if we have 50GB audit log file in spool directory then it is taking
> 4-5hr for the conversion and writing to HDFS.
> Also, we are observing below error logs
> {code:java}
> ERROR [AuditFileQueueSpool_hdfs_destWriter] provider.BaseAuditHandler: Error
> writing to log file.
> java.lang.RuntimeException: Overflow of newLength.
> smallBuffer.length=1073741824, nextElemLength=38
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.increaseBufferSpace(BytesColumnVector.java:311)
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:182)
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:207)
> at org.apache.ranger.audit.utils.ORCFileUtil.log(ORCFileUtil.java:143)
> at
> org.apache.ranger.audit.utils.RangerORCAuditWriter$1.run(RangerORCAuditWriter.java:77)
> at
> org.apache.ranger.audit.utils.RangerORCAuditWriter$1.run(RangerORCAuditWriter.java:73)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
> at
> org.apache.ranger.audit.provider.MiscUtil.executePrivilegedAction(MiscUtil.java:541)
> at
> org.apache.ranger.audit.utils.RangerORCAuditWriter.logAuditAsORC(RangerORCAuditWriter.java:73)
> at
> org.apache.ranger.audit.utils.RangerORCAuditWriter.logAsORC(RangerORCAuditWriter.java:159)
> at
> org.apache.ranger.audit.utils.RangerORCAuditWriter.log(RangerORCAuditWriter.java:112)
> at
> org.apache.ranger.audit.destination.HDFSAuditDestination.logJSON(HDFSAuditDestination.java:78)
> at
> org.apache.ranger.audit.destination.HDFSAuditDestination.log(HDFSAuditDestination.java:163)
> at
> org.apache.ranger.audit.queue.AuditFileQueueSpool.sendEvent(AuditFileQueueSpool.java:926)
> at
> org.apache.ranger.audit.queue.AuditFileQueueSpool.logEvent(AuditFileQueueSpool.java:913)
> at
> org.apache.ranger.audit.queue.AuditFileQueueSpool.runLogAudit(AuditFileQueueSpool.java:847)
> at
>
[jira] [Commented] (RANGER-4404) Audit to hdfs for orc format feature stabilisation
[
https://issues.apache.org/jira/browse/RANGER-4404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764823#comment-17764823
]
Bhavik Patel commented on RANGER-4404:
--
Also checking the ORC commit for json to orc conversion :
https://github.com/apache/orc/pull/95/files#diff-9ff11212778d6c7d528ab25dd85d7ea8d363468f587fbd36aa566740e986b4d6
> Audit to hdfs for orc format feature stabilisation
> --
>
> Key: RANGER-4404
> URL: https://issues.apache.org/jira/browse/RANGER-4404
> Project: Ranger
> Issue Type: Improvement
> Components: audit
>Affects Versions: 3.0.0, 2.4.0
>Reporter: Bhavik Patel
>Assignee: Bhavik Patel
>Priority: Major
>
> Currently if we have 50GB audit log file in spool directory then it is taking
> 4-5hr for the conversion and writing to HDFS.
> Also, we are observing below error logs
> {code:java}
> ERROR [AuditFileQueueSpool_hdfs_destWriter] provider.BaseAuditHandler: Error
> writing to log file.
> java.lang.RuntimeException: Overflow of newLength.
> smallBuffer.length=1073741824, nextElemLength=38
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.increaseBufferSpace(BytesColumnVector.java:311)
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:182)
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:207)
> at org.apache.ranger.audit.utils.ORCFileUtil.log(ORCFileUtil.java:143)
> at
> org.apache.ranger.audit.utils.RangerORCAuditWriter$1.run(RangerORCAuditWriter.java:77)
> at
> org.apache.ranger.audit.utils.RangerORCAuditWriter$1.run(RangerORCAuditWriter.java:73)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
> at
> org.apache.ranger.audit.provider.MiscUtil.executePrivilegedAction(MiscUtil.java:541)
> at
> org.apache.ranger.audit.utils.RangerORCAuditWriter.logAuditAsORC(RangerORCAuditWriter.java:73)
> at
> org.apache.ranger.audit.utils.RangerORCAuditWriter.logAsORC(RangerORCAuditWriter.java:159)
> at
> org.apache.ranger.audit.utils.RangerORCAuditWriter.log(RangerORCAuditWriter.java:112)
> at
> org.apache.ranger.audit.destination.HDFSAuditDestination.logJSON(HDFSAuditDestination.java:78)
> at
> org.apache.ranger.audit.destination.HDFSAuditDestination.log(HDFSAuditDestination.java:163)
> at
> org.apache.ranger.audit.queue.AuditFileQueueSpool.sendEvent(AuditFileQueueSpool.java:926)
> at
> org.apache.ranger.audit.queue.AuditFileQueueSpool.logEvent(AuditFileQueueSpool.java:913)
> at
> org.apache.ranger.audit.queue.AuditFileQueueSpool.runLogAudit(AuditFileQueueSpool.java:847)
> at
> org.apache.ranger.audit.queue.AuditFileQueueSpool.run(AuditFileQueueSpool.java:790)
> {code}
> hive-storage-api version upgrade(>=2.7.3) required to resolve the above error.
> Current version is 2.7.2
> cc: [~rmani] [~fateh288]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (RANGER-4404) Audit to hdfs for orc format feature stabilisation
[
https://issues.apache.org/jira/browse/RANGER-4404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764818#comment-17764818
]
Bhavik Patel commented on RANGER-4404:
--
Observed these 4 for loop so first thought of checking whether we can avoind
any or not?
{code:java}
1 for loop in AuditFileQueueSpool(method logEvent(BufferedReader br)) =>
converting line to AuditEventBase Object
1 for loop in HDFSAuditDestination(method log(Collection
events)) => converting event to List of String
1 for loop in RangerORCAuditWriter(method
getAuthzAuditEvents(Collection events)) => converting event to
AuthzAuditEvent Object
1 for loop in ORCFileUtil (method log(Writer writer,
Collection events) => Actul mapping/adding the event data to
ORC{code}
So to avoid all the 2 for loop I updated the main method in the ORCFileUtil
class and tried to convert 10GB file so after 25min I got OOM(I explicitly set
heap size to 20GB).
So now thinking to check some-other approach.
{color:#de350b}Note: Also, I directly tried to use apache ORC-tools to convert
4.7GB json .log file to .ORC and only it took only 3min.
{color}
> Audit to hdfs for orc format feature stabilisation
> --
>
> Key: RANGER-4404
> URL: https://issues.apache.org/jira/browse/RANGER-4404
> Project: Ranger
> Issue Type: Improvement
> Components: audit
>Affects Versions: 3.0.0, 2.4.0
>Reporter: Bhavik Patel
>Assignee: Bhavik Patel
>Priority: Major
>
> Currently if we have 50GB audit log file in spool directory then it is taking
> 4-5hr for the conversion and writing to HDFS.
> Also, we are observing below error logs
> {code:java}
> ERROR [AuditFileQueueSpool_hdfs_destWriter] provider.BaseAuditHandler: Error
> writing to log file.
> java.lang.RuntimeException: Overflow of newLength.
> smallBuffer.length=1073741824, nextElemLength=38
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.increaseBufferSpace(BytesColumnVector.java:311)
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:182)
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:207)
> at org.apache.ranger.audit.utils.ORCFileUtil.log(ORCFileUtil.java:143)
> at
> org.apache.ranger.audit.utils.RangerORCAuditWriter$1.run(RangerORCAuditWriter.java:77)
> at
> org.apache.ranger.audit.utils.RangerORCAuditWriter$1.run(RangerORCAuditWriter.java:73)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
> at
> org.apache.ranger.audit.provider.MiscUtil.executePrivilegedAction(MiscUtil.java:541)
> at
> org.apache.ranger.audit.utils.RangerORCAuditWriter.logAuditAsORC(RangerORCAuditWriter.java:73)
> at
> org.apache.ranger.audit.utils.RangerORCAuditWriter.logAsORC(RangerORCAuditWriter.java:159)
> at
> org.apache.ranger.audit.utils.RangerORCAuditWriter.log(RangerORCAuditWriter.java:112)
> at
> org.apache.ranger.audit.destination.HDFSAuditDestination.logJSON(HDFSAuditDestination.java:78)
> at
> org.apache.ranger.audit.destination.HDFSAuditDestination.log(HDFSAuditDestination.java:163)
> at
> org.apache.ranger.audit.queue.AuditFileQueueSpool.sendEvent(AuditFileQueueSpool.java:926)
> at
> org.apache.ranger.audit.queue.AuditFileQueueSpool.logEvent(AuditFileQueueSpool.java:913)
> at
> org.apache.ranger.audit.queue.AuditFileQueueSpool.runLogAudit(AuditFileQueueSpool.java:847)
> at
> org.apache.ranger.audit.queue.AuditFileQueueSpool.run(AuditFileQueueSpool.java:790)
> {code}
> hive-storage-api version upgrade(>=2.7.3) required to resolve the above error.
> Current version is 2.7.2
> cc: [~rmani] [~fateh288]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (RANGER-4404) Audit to hdfs for orc format feature stabilisation
[
https://issues.apache.org/jira/browse/RANGER-4404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bhavik Patel updated RANGER-4404:
-
Description:
Currently if we have 50GB audit log file in spool directory then it is taking
4-5hr for the conversion and writing to HDFS.
Also, we are observing below error logs
{code:java}
ERROR [AuditFileQueueSpool_hdfs_destWriter] provider.BaseAuditHandler: Error
writing to log file.
java.lang.RuntimeException: Overflow of newLength.
smallBuffer.length=1073741824, nextElemLength=38
at
org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.increaseBufferSpace(BytesColumnVector.java:311)
at
org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:182)
at
org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:207)
at org.apache.ranger.audit.utils.ORCFileUtil.log(ORCFileUtil.java:143)
at
org.apache.ranger.audit.utils.RangerORCAuditWriter$1.run(RangerORCAuditWriter.java:77)
at
org.apache.ranger.audit.utils.RangerORCAuditWriter$1.run(RangerORCAuditWriter.java:73)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
at
org.apache.ranger.audit.provider.MiscUtil.executePrivilegedAction(MiscUtil.java:541)
at
org.apache.ranger.audit.utils.RangerORCAuditWriter.logAuditAsORC(RangerORCAuditWriter.java:73)
at
org.apache.ranger.audit.utils.RangerORCAuditWriter.logAsORC(RangerORCAuditWriter.java:159)
at
org.apache.ranger.audit.utils.RangerORCAuditWriter.log(RangerORCAuditWriter.java:112)
at
org.apache.ranger.audit.destination.HDFSAuditDestination.logJSON(HDFSAuditDestination.java:78)
at
org.apache.ranger.audit.destination.HDFSAuditDestination.log(HDFSAuditDestination.java:163)
at
org.apache.ranger.audit.queue.AuditFileQueueSpool.sendEvent(AuditFileQueueSpool.java:926)
at
org.apache.ranger.audit.queue.AuditFileQueueSpool.logEvent(AuditFileQueueSpool.java:913)
at
org.apache.ranger.audit.queue.AuditFileQueueSpool.runLogAudit(AuditFileQueueSpool.java:847)
at
org.apache.ranger.audit.queue.AuditFileQueueSpool.run(AuditFileQueueSpool.java:790)
{code}
hive-storage-api version upgrade(>=2.7.3) required to resolve the above error.
Current version is 2.7.2
cc: [~rmani] [~fateh288]
was:
Currently if we have 50GB audit log file in spool directory then it is taking
4-5hr for the conversion and writing to HDFS.
Also, we are observing below error logs
{code:java}
ERROR [AuditFileQueueSpool_hdfs_destWriter] provider.BaseAuditHandler: Error
writing to log file.
java.lang.RuntimeException: Overflow of newLength.
smallBuffer.length=1073741824, nextElemLength=38
at
org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.increaseBufferSpace(BytesColumnVector.java:311)
at
org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:182)
at
org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:207)
at org.apache.ranger.audit.utils.ORCFileUtil.log(ORCFileUtil.java:143)
at
org.apache.ranger.audit.utils.RangerORCAuditWriter$1.run(RangerORCAuditWriter.java:77)
at
org.apache.ranger.audit.utils.RangerORCAuditWriter$1.run(RangerORCAuditWriter.java:73)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
at
org.apache.ranger.audit.provider.MiscUtil.executePrivilegedAction(MiscUtil.java:541)
at
org.apache.ranger.audit.utils.RangerORCAuditWriter.logAuditAsORC(RangerORCAuditWriter.java:73)
at
org.apache.ranger.audit.utils.RangerORCAuditWriter.logAsORC(RangerORCAuditWriter.java:159)
at
org.apache.ranger.audit.utils.RangerORCAuditWriter.log(RangerORCAuditWriter.java:112)
at
org.apache.ranger.audit.destination.HDFSAuditDestination.logJSON(HDFSAuditDestination.java:78)
at
org.apache.ranger.audit.destination.HDFSAuditDestination.log(HDFSAuditDestination.java:163)
at
org.apache.ranger.audit.queue.AuditFileQueueSpool.sendEvent(AuditFileQueueSpool.java:926)
at
org.apache.ranger.audit.queue.AuditFileQueueSpool.logEvent(AuditFileQueueSpool.java:913)
at
org.apache.ranger.audit.queue.AuditFileQueueSpool.runLogAudit(AuditFileQueueSpool.java:847)
at
org.apache.ranger.audit.queue.AuditFileQueueSpool.run(AuditFileQueueSpool.java:790)
{code}
hive-storage-api version upgrade(>=2.7.3) required to resolve the above error.
Current version is 2.7.2
> Audit to hdfs for orc format feature stabilisation
>
[jira] [Created] (RANGER-4404) Audit to hdfs for orc format feature stabilisation
Bhavik Patel created RANGER-4404:
Summary: Audit to hdfs for orc format feature stabilisation
Key: RANGER-4404
URL: https://issues.apache.org/jira/browse/RANGER-4404
Project: Ranger
Issue Type: Improvement
Components: audit
Affects Versions: 2.4.0, 3.0.0
Reporter: Bhavik Patel
Assignee: Bhavik Patel
Currently if we have 50GB audit log file in spool directory then it is taking
4-5hr for the conversion and writing to HDFS.
Also, we are observing below error logs
{code:java}
ERROR [AuditFileQueueSpool_hdfs_destWriter] provider.BaseAuditHandler: Error
writing to log file.
java.lang.RuntimeException: Overflow of newLength.
smallBuffer.length=1073741824, nextElemLength=38
at
org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.increaseBufferSpace(BytesColumnVector.java:311)
at
org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:182)
at
org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:207)
at org.apache.ranger.audit.utils.ORCFileUtil.log(ORCFileUtil.java:143)
at
org.apache.ranger.audit.utils.RangerORCAuditWriter$1.run(RangerORCAuditWriter.java:77)
at
org.apache.ranger.audit.utils.RangerORCAuditWriter$1.run(RangerORCAuditWriter.java:73)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
at
org.apache.ranger.audit.provider.MiscUtil.executePrivilegedAction(MiscUtil.java:541)
at
org.apache.ranger.audit.utils.RangerORCAuditWriter.logAuditAsORC(RangerORCAuditWriter.java:73)
at
org.apache.ranger.audit.utils.RangerORCAuditWriter.logAsORC(RangerORCAuditWriter.java:159)
at
org.apache.ranger.audit.utils.RangerORCAuditWriter.log(RangerORCAuditWriter.java:112)
at
org.apache.ranger.audit.destination.HDFSAuditDestination.logJSON(HDFSAuditDestination.java:78)
at
org.apache.ranger.audit.destination.HDFSAuditDestination.log(HDFSAuditDestination.java:163)
at
org.apache.ranger.audit.queue.AuditFileQueueSpool.sendEvent(AuditFileQueueSpool.java:926)
at
org.apache.ranger.audit.queue.AuditFileQueueSpool.logEvent(AuditFileQueueSpool.java:913)
at
org.apache.ranger.audit.queue.AuditFileQueueSpool.runLogAudit(AuditFileQueueSpool.java:847)
at
org.apache.ranger.audit.queue.AuditFileQueueSpool.run(AuditFileQueueSpool.java:790)
{code}
hive-storage-api version upgrade(>=2.7.3) required to resolve the above error.
Current version is 2.7.2
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (RANGER-4390) ORC audit fails with null pointer exception when filequeue buffer size and orc buffer size are not equal
[
https://issues.apache.org/jira/browse/RANGER-4390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762027#comment-17762027
]
Bhavik Patel commented on RANGER-4390:
--
I can see the same error when xasecure.audit.destination.hdfs.orc.buffersize >
xasecure.audit.destination.hdfs.batch.filequeue.filespool.buffer.size
> ORC audit fails with null pointer exception when filequeue buffer size and
> orc buffer size are not equal
>
>
> Key: RANGER-4390
> URL: https://issues.apache.org/jira/browse/RANGER-4390
> Project: Ranger
> Issue Type: Bug
> Components: Ranger
>Reporter: Fateh Singh
>Priority: Major
>
> Steps to reproduce:
> Change config values in unit test case testAuditFileQueueSpoolORC() in
> TestAuditQueue.java
> xasecure.audit.destination.hdfs.orc.buffersize=5
> xasecure.audit.destination.hdfs.batch.filequeue.filespool.buffer.size=10
> If both are same then error not observed
> Sample config from test case to reproduce issue:
> Configs:
> {code:java}
> {xasecure.audit.destination.hdfs.orc.buffersize=5,
> xasecure.audit.destination.hdfs.batch.filequeue.filespool.dir=target/spool,
> xasecure.audit.destination.hdfs.batch.queuetype=filequeue,
> xasecure.audit.destination.hdfs.batch.filequeue.filespool.buffer.size=10,
> xasecure.audit.destination.hdfs.batch.filequeue.filetype=orc,
> xasecure.audit.is.enabled=true,
> xasecure.audit.destination.hdfs.filename.format=%app-type%_ranger_audit.orc,
> xasecure.audit.destination.hdfs=enable,
> xasecure.audit.destination.hdfs.orc.stripesize=10,
> xasecure.audit.destination.hdfs.dir=target/testAuditFileQueueSpoolORC,
> xasecure.audit.destination.hdfs.orc.compression=none,
> xasecure.audit.destination.hdfs.batch.filequeue.filespool.file.rollover.sec=5}{code}
> Error logs:
> {code:java}
> 13:52:49.800 [AuditFileQueueSpool_hdfs_destWriter] ERROR
> org.apache.ranger.audit.provider.BaseAuditHandler - Error writing to log file.
> java.lang.NullPointerException: null
> at java.lang.System.arraycopy(Native Method)
> at org.apache.hadoop.io.Text.set(Text.java:225)
> at org.apache.orc.impl.StringRedBlackTree.add(StringRedBlackTree.java:59)
> at
> org.apache.orc.impl.writer.StringTreeWriter.writeBatch(StringTreeWriter.java:70)
> at
> org.apache.orc.impl.writer.StructTreeWriter.writeRootBatch(StructTreeWriter.java:56)
> at org.apache.orc.impl.WriterImpl.addRowBatch(WriterImpl.java:574)
> at org.apache.ranger.audit.utils.ORCFileUtil.log(ORCFileUtil.java:147)
> at
> org.apache.ranger.audit.utils.RangerORCAuditWriter$1.run(RangerORCAuditWriter.java:77)
> at
> org.apache.ranger.audit.utils.RangerORCAuditWriter$1.run(RangerORCAuditWriter.java:73)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1845)
> at
> org.apache.ranger.audit.provider.MiscUtil.executePrivilegedAction(MiscUtil.java:544)
> at
> org.apache.ranger.audit.utils.RangerORCAuditWriter.logAuditAsORC(RangerORCAuditWriter.java:73)
> at
> org.apache.ranger.audit.utils.RangerORCAuditWriter.logAsORC(RangerORCAuditWriter.java:159)
> at
> org.apache.ranger.audit.utils.RangerORCAuditWriter.log(RangerORCAuditWriter.java:112)
> at
> org.apache.ranger.audit.destination.HDFSAuditDestination.logJSON(HDFSAuditDestination.java:79)
> at
> org.apache.ranger.audit.destination.HDFSAuditDestination.log(HDFSAuditDestination.java:171)
> at
> org.apache.ranger.audit.queue.AuditFileQueueSpool.sendEvent(AuditFileQueueSpool.java:926)
> at
> org.apache.ranger.audit.queue.AuditFileQueueSpool.logEvent(AuditFileQueueSpool.java:904)
> at
> org.apache.ranger.audit.queue.AuditFileQueueSpool.runLogAudit(AuditFileQueueSpool.java:847)
> at
> org.apache.ranger.audit.queue.AuditFileQueueSpool.run(AuditFileQueueSpool.java:790)
> at java.lang.Thread.run(Thread.java:750) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (RANGER-4357) Enabling ORC audit for hdfs destination should not require 2 separate configurations
[ https://issues.apache.org/jira/browse/RANGER-4357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17760995#comment-17760995 ] Bhavik Patel commented on RANGER-4357: -- Can you mention the list of properties you have used for validation? Are getting the logs in spool directories with in orc formate or json format only? > Enabling ORC audit for hdfs destination should not require 2 separate > configurations > > > Key: RANGER-4357 > URL: https://issues.apache.org/jira/browse/RANGER-4357 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Fateh Singh >Assignee: Fateh Singh >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Currently there are 2 configs which will be required to enable orc i.e. > xasecure.audit.destination.hdfs.batch.filequeue.filetype=orc and config > xasecure.audit.destination.hdfs.filetype=orc > Ideally, we would want only 1 config to enable orc > xasecure.audit.destination.hdfs.batch.filequeue.filetype=orc. Also, orc can > only be enabled from filequeue, so above config seems more reasonable. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-4384) ORC audit for hdfs destination is too slow
[ https://issues.apache.org/jira/browse/RANGER-4384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavik Patel updated RANGER-4384: - Description: ORC audit for hdfs destination is too slow when we generate 60GB audit in a day and audits are not being able to written from spool directories to HDFS due to slowness. Below Configurations are set on the test cluster: orc.buffersize=10mb orc.stripesize=10mb filespool.buffer.size=10k file.rollover.sec=1day xasecure.audit.provider.queue.size=1mb default CC: [~rmani] [~fateh288] was: ORC audit for hdfs destination is too slow when we generate 60GB audit in a day and due to slowness it started writing to spool directory. Even audits are not able flush from spool directories to HDFS due to slowness. Below Configurations are set on the test cluster: orc.buffersize=10mb orc.stripesize=10mb filespool.buffer.size=10k file.rollover.sec=1day xasecure.audit.provider.queue.size=1mb default CC: [~rmani] [~fateh288] > ORC audit for hdfs destination is too slow > -- > > Key: RANGER-4384 > URL: https://issues.apache.org/jira/browse/RANGER-4384 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.3.0, 2.4.0 >Reporter: Bhavik Patel >Priority: Major > > ORC audit for hdfs destination is too slow when we generate 60GB audit in a > day and audits are not being able to written from spool directories to HDFS > due to slowness. > Below Configurations are set on the test cluster: > orc.buffersize=10mb > orc.stripesize=10mb > filespool.buffer.size=10k > file.rollover.sec=1day > xasecure.audit.provider.queue.size=1mb default > > CC: [~rmani] [~fateh288] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4384) ORC audit for hdfs destination is too slow
[ https://issues.apache.org/jira/browse/RANGER-4384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17760232#comment-17760232 ] Bhavik Patel commented on RANGER-4384: -- we don't have any such script, we are actually running the hbase jobs. > ORC audit for hdfs destination is too slow > -- > > Key: RANGER-4384 > URL: https://issues.apache.org/jira/browse/RANGER-4384 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.3.0, 2.4.0 >Reporter: Bhavik Patel >Priority: Major > > ORC audit for hdfs destination is too slow when we generate 60GB audit in a > day and due to slowness it started writing to spool directory. Even audits > are not able flush from spool directories to HDFS due to slowness. > Below Configurations are set on the test cluster: > orc.buffersize=10mb > orc.stripesize=10mb > filespool.buffer.size=10k > file.rollover.sec=1day > xasecure.audit.provider.queue.size=1mb default > > CC: [~rmani] [~fateh288] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4384) ORC audit for hdfs destination is too slow
Bhavik Patel created RANGER-4384: Summary: ORC audit for hdfs destination is too slow Key: RANGER-4384 URL: https://issues.apache.org/jira/browse/RANGER-4384 Project: Ranger Issue Type: Bug Components: Ranger Affects Versions: 2.4.0, 2.3.0 Reporter: Bhavik Patel ORC audit for hdfs destination is too slow when we generate 60GB audit in a day and due to slowness it started writing to spool directory. Even audits are not able flush from spool directories to HDFS due to slowness. Below Configurations are set on the test cluster: orc.buffersize=10mb orc.stripesize=10mb filespool.buffer.size=10k file.rollover.sec=1day xasecure.audit.provider.queue.size=1mb default CC: [~rmani] [~fateh288] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4357) Enabling ORC audit for hdfs destination should not require 2 separate configurations
[
https://issues.apache.org/jira/browse/RANGER-4357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17759434#comment-17759434
]
Bhavik Patel commented on RANGER-4357:
--
[~fateh288] thanks for reporting this.
Have you also validated for default .json format?
On my cluster Im observing file rollover is not effective for
hbaseRegion(specifically) server, are you also observing the same?
{code:java}
orc.buffersize=10mb
orc.stripesize=10mb
filespool.buffer.size=10k
file.rollover.sec=1day
xasecure.audit.provider.queue.size=1mb default {code}
With the above configuration Im observing logs are accumulating in the
filespool directory. Do I need to optimise any config?
> Enabling ORC audit for hdfs destination should not require 2 separate
> configurations
>
>
> Key: RANGER-4357
> URL: https://issues.apache.org/jira/browse/RANGER-4357
> Project: Ranger
> Issue Type: Improvement
> Components: Ranger
>Reporter: Fateh Singh
>Assignee: Fateh Singh
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently there are 2 configs which will be required to enable orc i.e.
> xasecure.audit.destination.hdfs.batch.filequeue.filetype=orc and config
> xasecure.audit.destination.hdfs.filetype=orc
> Ideally, we would want only 1 config to enable orc
> xasecure.audit.destination.hdfs.batch.filequeue.filetype=orc. Also, orc can
> only be enabled from filequeue, so above config seems more reasonable.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (RANGER-4196) Tomcat metrics collection
[
https://issues.apache.org/jira/browse/RANGER-4196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757796#comment-17757796
]
Bhavik Patel commented on RANGER-4196:
--
Thanks [~vikkumar] .
{code:java}
"KMS": {
"GET_CURRENT_KEY_COUNT": 0,
"DELETE_KEY_ELAPSED_TIME": 0,
"EEK_DECRYPT_ELAPSED_TIME": 0,
"GET_KEYS_METADATA_ELAPSED_TIME": 0,
"EEK_GENERATE_ELAPSED_TIME": 0,
"GET_CURRENT_KEY_ELAPSED_TIME": 0,
"EEK_REENCRYPT_ELAPSED_TIME": 0,
"KEY_CREATE_COUNT": 0,
"UNAUTHORIZED_CALLS_COUNT": 0,
"KEY_CREATE_ELAPSED_TIME": 0,
"GET_KEY_VERSION_COUNT": 0,
"ROLL_NEW_VERSION_ELAPSED_TIME": 0,
"REENCRYPT_EEK_BATCH_COUNT": 0,
"REENCRYPT_EEK_BATCH_ELAPSED_TIME": 0,
"GET_KEYS_METADATA_COUNT": 0,
"GET_KEY_VERSIONS_COUNT": 0,
"GET_KEY_VERSIONS_ELAPSED_TIME": 0,
"GET_KEYS_COUNT": 0,
"EEK_GENERATE_COUNT": 0,
"INVALIDATE_CACHE_COUNT": 0,
"GET_METADATA_COUNT": 0,
"REENCRYPT_EEK_BATCH_KEYS_COUNT": 0,
"EEK_REENCRYPT_COUNT": 0,
"UNAUTHENTICATED_CALLS_COUNT": 3,
"GET_KEY_VERSION_ELAPSED_TIME": 0,
"INVALIDATE_CACHE_ELAPSED_TIME": 0,
"ROLL_NEW_VERSION_COUNT": 0,
"EEK_DECRYPT_COUNT": 0,
"GET_KEYS_METADATA_KEYNAMES_COUNT": 0,
"DELETE_KEY_COUNT": 0,
"GET_KEYS_ELAPSED_TIME": 0,
"GET_METADATA_ELAPSED_TIME": 0,
"TOTAL_CALL_COUNT": 6
}, {code}
Regarding the above output, can you perform some operations than get the
metrics and then we can actually map operations with TOTAL_CALL_COUNT
> Tomcat metrics collection
> --
>
> Key: RANGER-4196
> URL: https://issues.apache.org/jira/browse/RANGER-4196
> Project: Ranger
> Issue Type: Improvement
> Components: Ranger
>Reporter: Vikas Kumar
>Assignee: Vikas Kumar
>Priority: Major
>
> Like "JVM mterics" and other application metrics, Tomcat related metrics
> would be very useful for following use cases:
> In case client gets "SoketTimeout" or Connection refused errors, having such
> metrics at server end will really help triaging the issue.
> It should contain following metrics:
> # maxAllowedConnection
> # currentActiveConnectionCount
> # ConnectionTimeout
> # acceptCount
> # maxContainerThreadsCount
> # activeContainerThreadsCount
> # minSpareThreadsCount // corePoolSize
> And other useful metrics.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (RANGER-4178) NoClassDefFoundError: org/apache/hadoop/hive/ql/exec/vector/ColumnVector
[ https://issues.apache.org/jira/browse/RANGER-4178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17756971#comment-17756971 ] Bhavik Patel commented on RANGER-4178: -- [~fateh288] / [~rmani] Log files from spool directory is not getting cleaned up. Are you observing the same behaviour ? > NoClassDefFoundError: org/apache/hadoop/hive/ql/exec/vector/ColumnVector > > > Key: RANGER-4178 > URL: https://issues.apache.org/jira/browse/RANGER-4178 > Project: Ranger > Issue Type: Bug > Components: audit >Affects Versions: 3.0.0, 2.2.0, 2.3.0 >Reporter: Bhavik Patel >Assignee: Bhavik Patel >Priority: Critical > Attachments: > 0001-RANGER-4178-NoClassDefFoundError-org-apache-hadoop-h.patch, > 0001-RANGER-4178-Only-xasecure.audit.destination.hdfs.bat.patch > > > Observed below error when enabled audit type as ORC format. > NoClassDefFoundError: org/apache/hadoop/hive/ql/exec/vector/ColumnVector > https://issues.apache.org/jira/browse/RANGER-1837 > https://issues.apache.org/jira/browse/RANGER-3235 > cc: [~rmani] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4178) NoClassDefFoundError: org/apache/hadoop/hive/ql/exec/vector/ColumnVector
[ https://issues.apache.org/jira/browse/RANGER-4178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755423#comment-17755423 ] Bhavik Patel commented on RANGER-4178: -- [~fateh288] Sorry for the delay. My review board login issue is resolved now. which issue will resolve your patch? > NoClassDefFoundError: org/apache/hadoop/hive/ql/exec/vector/ColumnVector > > > Key: RANGER-4178 > URL: https://issues.apache.org/jira/browse/RANGER-4178 > Project: Ranger > Issue Type: Bug > Components: audit >Affects Versions: 3.0.0, 2.2.0, 2.3.0 >Reporter: Bhavik Patel >Assignee: Bhavik Patel >Priority: Critical > Attachments: > 0001-RANGER-4178-NoClassDefFoundError-org-apache-hadoop-h.patch, > 0001-RANGER-4178-Only-xasecure.audit.destination.hdfs.bat.patch > > > Observed below error when enabled audit type as ORC format. > NoClassDefFoundError: org/apache/hadoop/hive/ql/exec/vector/ColumnVector > https://issues.apache.org/jira/browse/RANGER-1837 > https://issues.apache.org/jira/browse/RANGER-3235 > cc: [~rmani] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4178) NoClassDefFoundError: org/apache/hadoop/hive/ql/exec/vector/ColumnVector
[ https://issues.apache.org/jira/browse/RANGER-4178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744422#comment-17744422 ] Bhavik Patel commented on RANGER-4178: -- I'm not able to login to review board due to some issue. [~fateh288] : pom changes in hive-agent.xml missing ==> hive component already have these jar in it classpath so it's not required. Once some one review then we can get it merge. > NoClassDefFoundError: org/apache/hadoop/hive/ql/exec/vector/ColumnVector > > > Key: RANGER-4178 > URL: https://issues.apache.org/jira/browse/RANGER-4178 > Project: Ranger > Issue Type: Bug > Components: audit >Affects Versions: 3.0.0, 2.2.0, 2.3.0 >Reporter: Bhavik Patel >Assignee: Bhavik Patel >Priority: Critical > Attachments: > 0001-RANGER-4178-NoClassDefFoundError-org-apache-hadoop-h.patch, > 0001-RANGER-4178-Only-xasecure.audit.destination.hdfs.bat.patch > > > Observed below error when enabled audit type as ORC format. > NoClassDefFoundError: org/apache/hadoop/hive/ql/exec/vector/ColumnVector > https://issues.apache.org/jira/browse/RANGER-1837 > https://issues.apache.org/jira/browse/RANGER-3235 > cc: [~rmani] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4196) Tomcat metrics collection
[ https://issues.apache.org/jira/browse/RANGER-4196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17737457#comment-17737457 ] Bhavik Patel commented on RANGER-4196: -- [~vikkumar] are you working on this? > Tomcat metrics collection > -- > > Key: RANGER-4196 > URL: https://issues.apache.org/jira/browse/RANGER-4196 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Vikas Kumar >Assignee: Vikas Kumar >Priority: Major > > Like "JVM mterics" and other application metrics, Tomcat related metrics > would be very useful for following use cases: > In case client gets "SoketTimeout" or Connection refused errors, having such > metrics at server end will really help triaging the issue. > It should contain following metrics: > # maxAllowedConnection > # currentActiveConnectionCount > # ConnectionTimeout > # acceptCount > # maxContainerThreadsCount > # activeContainerThreadsCount > # minSpareThreadsCount // corePoolSize > And other useful metrics. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (RANGER-4217) RANGER-1837 (enabling ORC audit logs) does not work
[ https://issues.apache.org/jira/browse/RANGER-4217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavik Patel resolved RANGER-4217. -- Resolution: Duplicate Review request available at : https://reviews.apache.org/r/74387/ > RANGER-1837 (enabling ORC audit logs) does not work > --- > > Key: RANGER-4217 > URL: https://issues.apache.org/jira/browse/RANGER-4217 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Fateh Singh >Assignee: Fateh Singh >Priority: Major > > Various bugs need to be addressed and documentation to enable ORC Audit logs > needs to be updated -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (RANGER-4201) Hbase master can't start due to ranger-hbase-plugin jersey jar conflict
[
https://issues.apache.org/jira/browse/RANGER-4201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17715721#comment-17715721
]
Bhavik Patel edited comment on RANGER-4201 at 4/24/23 11:18 AM:
Kindly check : https://issues.apache.org/jira/browse/RANGER-4176
We are not observing any such error, can you compare the plugins lib tree with
below.
{code:java}
total 40372
-rw-r--r--. 1 root root 8632395 Apr 20 06:54 eclipselink-2.5.2.jar
-rw-r--r--. 1 root root 13052737 Apr 20 06:54 elasticsearch-7.10.2.jar
-rw-r--r--. 1 root root 60064 Apr 20 06:54 elasticsearch-core-7.10.2.jar
-rw-r--r--. 1 root root 66224 Apr 20 06:54
elasticsearch-rest-client-7.10.2.jar
-rw-r--r--. 1 root root 1682704 Apr 20 06:54
elasticsearch-rest-high-level-client-7.10.2.jar
-rw-r--r--. 1 root root 142972 Apr 20 06:54 elasticsearch-x-content-7.10.2.jar
-rw-r--r--. 1 root root 4179 Apr 20 06:54 gethostname4j-0.0.2.jar
-rw-r--r--. 1 root root 2747878 Apr 20 06:54 guava-27.0-jre.jar
-rw-r--r--. 1 root root 1193515 Apr 20 06:54 hppc-0.8.0.jar
-rw-r--r--. 1 root root 179964 Apr 20 06:54 httpasyncclient-4.1.4.jar
-rw-r--r--. 1 root root 41790 Apr 20 06:54 httpmime-4.5.13.jar
-rw-r--r--. 1 root root 162126 Apr 20 06:54 javax.persistence-2.1.0.jar
-rw-r--r--. 1 root root 133824 Apr 20 06:54 jersey-client-1.19.3.jar
-rw-r--r--. 1 root root 434518 Apr 20 06:54 jersey-core-1.19.3.jar
-rw-r--r--. 1 root root 704349 Apr 20 06:54 jersey-server-1.19.3.jar
-rw-r--r--. 1 root root 327914 Apr 20 06:54 jetty-client-9.4.49.v20220914.jar
-rw-r--r--. 1 root root 1488769 Apr 20 06:54 jna-5.2.0.jar
-rw-r--r--. 1 root root 2527218 Apr 20 06:54 jna-platform-5.2.0.jar
-rw-r--r--. 1 root root 643778 Apr 20 06:54 joda-time-2.10.6.jar
-rw-r--r--. 1 root root 63705 Apr 20 06:54 lang-mustache-client-7.10.2.jar
-rw-r--r--. 1 root root 3300764 Apr 20 06:54 lucene-core-8.4.0.jar
-rw-r--r--. 1 root root 101913 Apr 20 06:54
ranger-hbase-plugin-2.1.0.3.2.2.4-1.jar
-rw-r--r--. 1 root root 210461 Apr 20 06:54
ranger-plugins-audit-2.1.0.3.2.2.4-1.jar
-rw-r--r--. 1 root root 896800 Apr 20 06:54
ranger-plugins-common-2.1.0.3.2.2.4-1.jar
-rw-r--r--. 1 root root 20368 Apr 20 06:54
ranger-plugins-cred-2.1.0.3.2.2.4-1.jar
-rw-r--r--. 1 root root 82835 Apr 20 06:54 rank-eval-client-7.10.2.jar
-rw-r--r--. 1 root root 2391561 Apr 20 06:54 solr-solrj-8.11.2.jar {code}
was (Author: bpatel):
This is already fixed and available in RANAGER-3.0.0
Kindly check : https://issues.apache.org/jira/browse/RANGER-4176
> Hbase master can't start due to ranger-hbase-plugin jersey jar conflict
>
>
> Key: RANGER-4201
> URL: https://issues.apache.org/jira/browse/RANGER-4201
> Project: Ranger
> Issue Type: Bug
> Components: Ranger
>Affects Versions: 2.3.0, 2.4.0
>Reporter: caijialiang
>Priority: Major
>
> hbase version 2.4.13
> ranger version 2.4
> hbase master can't start after installed ranger-hbase-plugin
>
> {code:java}
> ERROR [master/gs-server-13481:16000:becomeActiveMaster] master.HMaster: *
> ABORTING master gs-server-13481,16000,1681377561702: The coprocessor
> org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor threw
> java.lang.ExceptionInIniti
> alizerError *
> java.lang.ExceptionInInitializerError
> at
> com.sun.jersey.core.spi.factory.MessageBodyFactory.initReaders(MessageBodyFactory.java:182)
> at
> com.sun.jersey.core.spi.factory.MessageBodyFactory.initReaders(MessageBodyFactory.java:175)
> at
> com.sun.jersey.core.spi.factory.MessageBodyFactory.init(MessageBodyFactory.java:162)
> at com.sun.jersey.api.client.Client.init(Client.java:343)
> at com.sun.jersey.api.client.Client.access$000(Client.java:119)
> at com.sun.jersey.api.client.Client$1.f(Client.java:192)
> at com.sun.jersey.api.client.Client$1.f(Client.java:188)
> at com.sun.jersey.spi.inject.Errors.processWithErrors(Errors.java:193)
> at com.sun.jersey.api.client.Client.(Client.java:188)
> at com.sun.jersey.api.client.Client.(Client.java:171)
> at com.sun.jersey.api.client.Client.create(Client.java:683)
> at
> org.apache.ranger.plugin.util.RangerRESTClient.buildClient(RangerRESTClient.java:228)
> at
> org.apache.ranger.plugin.util.RangerRESTClient.getClient(RangerRESTClient.java:193)
> at
> org.apache.ranger.plugin.util.RangerRESTClient.get(RangerRESTClient.java:473)
> at
> org.apache.ranger.admin.client.RangerAdminRESTClient.getRangerRolesDownloadResponse(RangerAdminRESTClient.java:1340)
> at
> org.apache.ranger.admin.client.RangerAdminRESTClient.getRolesIfUpdatedWithCred(RangerAdminRESTClient.java:1202)
> at
>
[jira] [Reopened] (RANGER-4201) Hbase master can't start due to ranger-hbase-plugin jersey jar conflict
[
https://issues.apache.org/jira/browse/RANGER-4201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bhavik Patel reopened RANGER-4201:
--
> Hbase master can't start due to ranger-hbase-plugin jersey jar conflict
>
>
> Key: RANGER-4201
> URL: https://issues.apache.org/jira/browse/RANGER-4201
> Project: Ranger
> Issue Type: Bug
> Components: Ranger
>Affects Versions: 2.3.0, 2.4.0
>Reporter: caijialiang
>Priority: Major
>
> hbase version 2.4.13
> ranger version 2.4
> hbase master can't start after installed ranger-hbase-plugin
>
> {code:java}
> ERROR [master/gs-server-13481:16000:becomeActiveMaster] master.HMaster: *
> ABORTING master gs-server-13481,16000,1681377561702: The coprocessor
> org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor threw
> java.lang.ExceptionInIniti
> alizerError *
> java.lang.ExceptionInInitializerError
> at
> com.sun.jersey.core.spi.factory.MessageBodyFactory.initReaders(MessageBodyFactory.java:182)
> at
> com.sun.jersey.core.spi.factory.MessageBodyFactory.initReaders(MessageBodyFactory.java:175)
> at
> com.sun.jersey.core.spi.factory.MessageBodyFactory.init(MessageBodyFactory.java:162)
> at com.sun.jersey.api.client.Client.init(Client.java:343)
> at com.sun.jersey.api.client.Client.access$000(Client.java:119)
> at com.sun.jersey.api.client.Client$1.f(Client.java:192)
> at com.sun.jersey.api.client.Client$1.f(Client.java:188)
> at com.sun.jersey.spi.inject.Errors.processWithErrors(Errors.java:193)
> at com.sun.jersey.api.client.Client.(Client.java:188)
> at com.sun.jersey.api.client.Client.(Client.java:171)
> at com.sun.jersey.api.client.Client.create(Client.java:683)
> at
> org.apache.ranger.plugin.util.RangerRESTClient.buildClient(RangerRESTClient.java:228)
> at
> org.apache.ranger.plugin.util.RangerRESTClient.getClient(RangerRESTClient.java:193)
> at
> org.apache.ranger.plugin.util.RangerRESTClient.get(RangerRESTClient.java:473)
> at
> org.apache.ranger.admin.client.RangerAdminRESTClient.getRangerRolesDownloadResponse(RangerAdminRESTClient.java:1340)
> at
> org.apache.ranger.admin.client.RangerAdminRESTClient.getRolesIfUpdatedWithCred(RangerAdminRESTClient.java:1202)
> at
> org.apache.ranger.admin.client.RangerAdminRESTClient.getRolesIfUpdated(RangerAdminRESTClient.java:167)
> at
> org.apache.ranger.plugin.util.RangerRolesProvider.loadUserGroupRolesFromAdmin(RangerRolesProvider.java:183)
> at
> org.apache.ranger.plugin.util.RangerRolesProvider.loadUserGroupRoles(RangerRolesProvider.java:123)
> at
> org.apache.ranger.plugin.util.PolicyRefresher.loadRoles(PolicyRefresher.java:495)
> at
> org.apache.ranger.plugin.util.PolicyRefresher.startRefresher(PolicyRefresher.java:144)
> at
> org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:245)
> at
> org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor.start(RangerAuthorizationCoprocessor.java:1120)
> at
> org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor.start(RangerAuthorizationCoprocessor.java:160)
> at
> org.apache.hadoop.hbase.coprocessor.BaseEnvironment.startup(BaseEnvironment.java:69)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.checkAndLoadInstance(CoprocessorHost.java:285)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadSystemCoprocessors(CoprocessorHost.java:171)
> at
> org.apache.hadoop.hbase.master.MasterCoprocessorHost.(MasterCoprocessorHost.java:155)
> at
> org.apache.hadoop.hbase.master.HMaster.initializeCoprocessorHost(HMaster.java:3870)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:902)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2175)
> at
> org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:520)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> org.glassfish.jersey.internal.RuntimeDelegateImpl
> at
> javax.ws.rs.ext.RuntimeDelegate.findDelegate(RuntimeDelegate.java:130)
> at
> javax.ws.rs.ext.RuntimeDelegate.getInstance(RuntimeDelegate.java:97)
> at javax.ws.rs.core.MediaType.valueOf(MediaType.java:172)
> at com.sun.jersey.core.header.MediaTypes.(MediaTypes.java:65)
> ... 33 more
> Caused by: java.lang.ClassNotFoundException:
> org.glassfish.jersey.internal.RuntimeDelegateImpl
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at
[jira] [Resolved] (RANGER-4201) Hbase master can't start due to ranger-hbase-plugin jersey jar conflict
[
https://issues.apache.org/jira/browse/RANGER-4201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bhavik Patel resolved RANGER-4201.
--
Resolution: Duplicate
This is already fixed and available in RANAGER-3.0.0
Kindly check : https://issues.apache.org/jira/browse/RANGER-4176
> Hbase master can't start due to ranger-hbase-plugin jersey jar conflict
>
>
> Key: RANGER-4201
> URL: https://issues.apache.org/jira/browse/RANGER-4201
> Project: Ranger
> Issue Type: Bug
> Components: Ranger
>Affects Versions: 2.3.0, 2.4.0
>Reporter: caijialiang
>Priority: Major
>
> hbase version 2.4.13
> ranger version 2.4
> hbase master can't start after installed ranger-hbase-plugin
>
> {code:java}
> ERROR [master/gs-server-13481:16000:becomeActiveMaster] master.HMaster: *
> ABORTING master gs-server-13481,16000,1681377561702: The coprocessor
> org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor threw
> java.lang.ExceptionInIniti
> alizerError *
> java.lang.ExceptionInInitializerError
> at
> com.sun.jersey.core.spi.factory.MessageBodyFactory.initReaders(MessageBodyFactory.java:182)
> at
> com.sun.jersey.core.spi.factory.MessageBodyFactory.initReaders(MessageBodyFactory.java:175)
> at
> com.sun.jersey.core.spi.factory.MessageBodyFactory.init(MessageBodyFactory.java:162)
> at com.sun.jersey.api.client.Client.init(Client.java:343)
> at com.sun.jersey.api.client.Client.access$000(Client.java:119)
> at com.sun.jersey.api.client.Client$1.f(Client.java:192)
> at com.sun.jersey.api.client.Client$1.f(Client.java:188)
> at com.sun.jersey.spi.inject.Errors.processWithErrors(Errors.java:193)
> at com.sun.jersey.api.client.Client.(Client.java:188)
> at com.sun.jersey.api.client.Client.(Client.java:171)
> at com.sun.jersey.api.client.Client.create(Client.java:683)
> at
> org.apache.ranger.plugin.util.RangerRESTClient.buildClient(RangerRESTClient.java:228)
> at
> org.apache.ranger.plugin.util.RangerRESTClient.getClient(RangerRESTClient.java:193)
> at
> org.apache.ranger.plugin.util.RangerRESTClient.get(RangerRESTClient.java:473)
> at
> org.apache.ranger.admin.client.RangerAdminRESTClient.getRangerRolesDownloadResponse(RangerAdminRESTClient.java:1340)
> at
> org.apache.ranger.admin.client.RangerAdminRESTClient.getRolesIfUpdatedWithCred(RangerAdminRESTClient.java:1202)
> at
> org.apache.ranger.admin.client.RangerAdminRESTClient.getRolesIfUpdated(RangerAdminRESTClient.java:167)
> at
> org.apache.ranger.plugin.util.RangerRolesProvider.loadUserGroupRolesFromAdmin(RangerRolesProvider.java:183)
> at
> org.apache.ranger.plugin.util.RangerRolesProvider.loadUserGroupRoles(RangerRolesProvider.java:123)
> at
> org.apache.ranger.plugin.util.PolicyRefresher.loadRoles(PolicyRefresher.java:495)
> at
> org.apache.ranger.plugin.util.PolicyRefresher.startRefresher(PolicyRefresher.java:144)
> at
> org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:245)
> at
> org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor.start(RangerAuthorizationCoprocessor.java:1120)
> at
> org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor.start(RangerAuthorizationCoprocessor.java:160)
> at
> org.apache.hadoop.hbase.coprocessor.BaseEnvironment.startup(BaseEnvironment.java:69)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.checkAndLoadInstance(CoprocessorHost.java:285)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadSystemCoprocessors(CoprocessorHost.java:171)
> at
> org.apache.hadoop.hbase.master.MasterCoprocessorHost.(MasterCoprocessorHost.java:155)
> at
> org.apache.hadoop.hbase.master.HMaster.initializeCoprocessorHost(HMaster.java:3870)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:902)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2175)
> at
> org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:520)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> org.glassfish.jersey.internal.RuntimeDelegateImpl
> at
> javax.ws.rs.ext.RuntimeDelegate.findDelegate(RuntimeDelegate.java:130)
> at
> javax.ws.rs.ext.RuntimeDelegate.getInstance(RuntimeDelegate.java:97)
> at javax.ws.rs.core.MediaType.valueOf(MediaType.java:172)
> at com.sun.jersey.core.header.MediaTypes.(MediaTypes.java:65)
> ... 33 more
> Caused by: java.lang.ClassNotFoundException:
>
