Madhan Neethiraj created RANGER-4883:
----------------------------------------
Summary: audit to HDFS fails with error:
org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for scheme
"hdfs"
Key: RANGER-4883
URL: https://issues.apache.org/jira/browse/RANGER-4883
Project: Ranger
Issue Type: Bug
Components: audit, plugins
Affects Versions: 3.0.0, 2.5.0
Reporter: Madhan Neethiraj
Assignee: Madhan Neethiraj
Writing audit log to HDFS fails with the following error in Kafka and Knox
plugins:
Kafka plugin:
{noformat}
[2024-08-01 01:48:50,242] ERROR Exception encountered while writing audits to
HDFS! (org.apache.ranger.audit.utils.RangerJSONAuditWriter)
org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for scheme
"hdfs"
at
org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:3443)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3466)
at org.apache.hadoop.fs.FileSystem.access$300(FileSystem.java:174)
at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3574)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3521)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:540)
at
org.apache.ranger.audit.utils.AbstractRangerAuditWriter.createFileSystemFolders(AbstractRangerAuditWriter.java:97)
{noformat}
Knox plugin:
{noformat}
01:52:21.193 [org.apache.ranger.audit.queue.AuditBatchQueue1] ERROR
org.apache.ranger.audit.utils.RangerJSONAuditWriter - Exception encountered
while writing audits to HDFS!
org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for scheme
"hdfs"
at
org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:3443)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3466)
at org.apache.hadoop.fs.FileSystem.access$300(FileSystem.java:174)
at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3574)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3521)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:540)
at
org.apache.ranger.audit.utils.AbstractRangerAuditWriter.createFileSystemFolders(AbstractRangerAuditWriter.java:97)
{noformat}
This is due to the plugins packaging missing library
{{{}hadoop-hdfs-client{}}}, which includes the necessary entry to handle "hdfs"
file scheme in META-INF/services/
org.apache.hadoop.hdfs.fs.FileSystem.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
