[
https://issues.apache.org/jira/browse/HDDS-98?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16591724#comment-16591724
]
Dinesh Chitlangia edited comment on HDDS-98 at 8/24/18 2:19 PM:
----------------------------------------------------------------
[~jnp] Thank you for a detailed feedback.
In this implementation, we are using the StructuredDataMessage (conforms to RFC
5424) for logging.
Its basic structure is:
{quote}{{{color:#59afe1}type{color} {color:#654982}ID{color} dataMap
{color:#f79232}message{color} throwable}}
{quote}
* type - Customer action types we have defined for ozone like CREATE_VOLUME,
CREATE_BUCKET etc
* ID - I can log a string "username IPAddress"
* dataMap - It is a key-value map object that we prepare for logging and can
thus add as many key-value pairs as want. So this addresses the extensibility
aspect. We cannot log username/IPAddress as key-value pair in the dataMap
because the dataMap will sort by key while printing and currently there is no
API to turn off sorting.
* message - any specific message we may want to pass, it can also be left
blank. We decided to pass this with only two values SUCCESS and FAILURE
depending upon the result. In future when we build a log parser, this will be
handy for querying various combinations.
* throwable - if any exception was thrown in the actual operation, we can log
it. This is totally configurable via properties file to log - x lines of
exception or error message or not log at all.
Thus, we end up with a log entry like the one below.
{quote}{{2018-08-24 04:57:28,685 | INFO | OMAudit |
{color:#59afe1}CREATE_VOLUME{color} [{color:#654982}xiaoyu/172.18.0.3
{color}{color:#14892c}admin="xiaoyu" creationTime="0" owner="xiaoyu"
quotaInBytes="1099511627776" volume="xmen"{color}]
{color:#f79232}SUCCESS{color} |}}
{quote}
> I can definitely change the "username/IP" to "username IP" and ensure this
> is at top level by logging it as ID.
> Extensibility has been addressed by {color:#14892c}dataMap{color}
> With this structure, the parsing will likely be easier as the actual data in
> enclosed in [ ]. Even if there were nested [ ] inside this, I feel it will
> still be easy to parse when we write a parser.
> Given the above structure, unfortunately, we won't be able to log like the
> example you provided
Please do let me know if there are further improvements that can be made and
thank you once again for a detailed feedback. :)
was (Author: dineshchitlangia):
[~jnp] Thank you for a detailed feedback.
In this implementation, we are using the StructuredDataMessage (conforms to RFC
5424) for logging.
Its basic structure is:
{quote}{{{color:#59afe1}type{color} {color:#654982}ID{color} dataMap
{color:#f79232}message{color} throwable}}
{quote}
* type - Customer action types we have defined for ozone like CREATE_VOLUME,
CREATE_BUCKET etc
* ID - I can log a string "username IPAddress"
* dataMap - It is a key-value map object that we prepare for logging and can
thus add as many key-value pairs as want. So this addresses the extensibility
aspect. We cannot log username/IPAddress as key-value pair in the dataMap
because the dataMap will sort by key while printing and currently there is no
API to turn off sorting.
* message - any specific message we may want to pass, it can also be left
blank. We decided to pass this with only two values SUCCESS and FAILURE
depending upon the result. In future when we build a log parser, this will be
handy for querying various combinations.
* throwable - if any exception was thrown in the actual operation, we can log
it. This is totally configurable via properties file to log - x lines of
exception or error message or not log at all.
Thus, we end up with a log entry like the one below.
{quote}{{2018-08-24 04:57:28,685 | INFO | OMAudit |
{color:#59afe1}CREATE_VOLUME{color} [{color:#654982}xiaoyu/172.18.0.3
{color}{color:#14892c}admin="xiaoyu" creationTime="0" owner="xiaoyu"
quotaInBytes="1099511627776" volume="xmen"{color}]
{color:#f79232}SUCCESS{color} |}}
{quote}
> I can definitely change the "username/IP" to "username IP" and ensure this
> is at top level by logging it as ID.
> Extensibility has been addressed by {color:#14892c}dataMap{color}
{color:#14892c}> With this structure, the parsing will likely be easier as the
actual data in enclosed in [ ]. Even if there were nested [ ] inside this, I
feel it will still be easy to parse when we write a parser{color}
> Given the above structure, unfortunately, we won't be able to log like the
> example you provided
Please do let me know if there are further improvements that can be made and
thank you once again for a detailed feedback. :)
> Adding Ozone Manager Audit Log
> ------------------------------
>
> Key: HDDS-98
> URL: https://issues.apache.org/jira/browse/HDDS-98
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Reporter: Xiaoyu Yao
> Assignee: Dinesh Chitlangia
> Priority: Major
> Labels: Logging, audit
> Fix For: 0.2.1
>
> Attachments: HDDS-98.001.patch, HDDS-98.002.patch, HDDS-98.003.patch,
> HDDS-98.004.patch, audit.log, log4j2.properties
>
>
> This ticket is opened to add ozone manager's audit log.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
