Hi, We are working on the implementation of the 2.4 released version of Apache Ranger in our cluster which includes 2 Hdfs namenodes. About audit logs, we are trying to use Opensearch which is likely to be compatible with elastic search version 7.10.2.
We mainly have two errors about the implementation of a HDFS service manager and we didn't find a way to correct them. *First error * The first one appears each time I use 2 URLs in the "Namenode URL" property in my HDFS service manager, using the indicated syntax on the WebUI such as below, and that I want to write a resource path and so to use the autocompletion. [image: image.png] *The syntax shown in the WebUI* Here is the error visible in the Apache Ranger log file : > > 2023-08-10 13:25:21,961 [timed-executor-pool-0] WARN > [FileSystem.java:3418] Failed to initialize fileystem hdfs:// > namemode01.test.ch:8020,hdfs://namenode02.test.ch:8020: > java.io.IOException: Incomplete HDFS URI, no host: hdfs:// > namemode01.test.ch:8020,hdfs://namenode02.test.ch:8020 > 2023-08-10 13:25:21,961 [timed-executor-pool-0] DEBUG > [FileSystem.java:3420] Failed to initialize fileystem > java.io.IOException: Incomplete HDFS URI, no host: hdfs:// > namemode01.test.ch:8020,hdfs://namenode02.test.ch:8020 > at > org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:181) > at > org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3414) > When I only put the active namenode of the moment in the property, it seems to work perfectly fine. So it's really about using 2 URLs. *Did you manage to make this feature work on your side ?* *Second error * The second one is linked to audit logs. Here is the template of the properties I use about the audit logs in my HDFS plugin. > XAAUDIT.ELASTICSEARCH.ENABLE=true > XAAUDIT.ELASTICSEARCH.URL=ranger-audit.test.ch > <http://ranger-dev-audit.test.ch> > XAAUDIT.ELASTICSEARCH.USER=ranger_internal > XAAUDIT.ELASTICSEARCH.PASSWORD=password > XAAUDIT.ELASTICSEARCH.INDEX=ranger_auditlog > XAAUDIT.ELASTICSEARCH.PORT=443 > XAAUDIT.ELASTICSEARCH.PROTOCOL=https > I wanted as a first try to use an internal user of my Opensearch instance (called ranger_internal), but it leads to this error : > <2023-08-16T17:12:16.107+0200> <ERROR> > <org.apache.ranger.audit.destination.ElasticSearchAuditDestination>: <Can't > connect to ElasticSearch server: User:ranger_internal, > https://ranger-audit-test.ch:443/ranger_auditlog> > java.lang.BootstrapMethodError: call site initialization exception > at java.lang.invoke.CallSite.makeSite(CallSite.java:341) > at > java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:307) > > at > java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:297) > > at > org.elasticsearch.client.RestClient.convertResponse(RestClient.java:312) > at > org.elasticsearch.client.RestClient.performRequest(RestClient.java:296) > at > org.elasticsearch.client.RestClient.performRequest(RestClient.java:270) > at > org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1632) > > I've tried to do some curl and my internal user had permissions to check the index. *Has any of you tried to use Opensearch ? Did it work ?* Cheers, Clément Lucas
