[
https://issues.apache.org/jira/browse/HDFS-12746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16225750#comment-16225750
]
Anu Engineer commented on HDFS-12746:
-------------------------------------
bq. I'm not sure about machine readable. Did you have something specific in
mind?
[~xkrogen] I was thinking about just logging the protobuf itself. Many of the
considerations of the original log format ( See this excellent blog from [~aw],
https://effectivemachines.com/2017/03/08/unofficial-history-of-the-hdfs-audit-log/
) for NN might not hold for data nodes (since there are so many of them,
searching via grep might not be very practical) and the representation is
concise with protobuf. Another advantage with protobuf is upgrades are
generally easy; even if we add a field.
The downside is that we will have to ship a library/tool to export the data
node logs to something like JSON/Text.
> DataNode Audit Logger
> ---------------------
>
> Key: HDFS-12746
> URL: https://issues.apache.org/jira/browse/HDFS-12746
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode, logging
> Reporter: Erik Krogen
>
> I would like to discuss adding in an audit logger for the Datanodes. We have
> audit logging on pretty much all other components: Namenode, ResourceManager,
> NodeManager. It seems the DN should have a similar concept to log, at
> minimum, all block reads/writes. I think all of the interesting information
> does already appear in the DN logs at INFO level but it would be nice to have
> a specific audit class that this gets logged through, a la {{RMAuditLogger}}
> and {{NMAuditLogger}}, to enable special handling.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
