[ https://issues.apache.org/jira/browse/HDDS-98?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16591724#comment-16591724 ]
Dinesh Chitlangia edited comment on HDDS-98 at 8/24/18 2:16 PM: ---------------------------------------------------------------- [~jnp] Thank you for a detailed feedback. In this implementation, we are using the StructuredDataMessage (conforms to RFC 5424) for logging. Its basic structure is: {quote}{{{color:#59afe1}type{color} {color:#654982}ID{color} dataMap {color:#f79232}message{color} throwable}} {quote} * type - Customer action types we have defined for ozone like CREATE_VOLUME, CREATE_BUCKET etc * ID - I can log a string "username IPAddress" * dataMap - It is a key-value map object that we prepare for logging and can thus add as many key-value pairs as want. So this addresses the extensibility aspect. We cannot log username/IPAddress as key-value pair in the dataMap because the dataMap will sort by key while printing and currently there is no API to turn off sorting. * message - any specific message we may want to pass, it can also be left blank. We decided to pass this with only two values SUCCESS and FAILURE depending upon the result. In future when we build a log parser, this will be handy for querying various combinations. * throwable - if any exception was thrown in the actual operation, we can log it. This is totally configurable via properties file to log - x lines of exception or error message or not log at all. Thus, we end up with a log entry like the one below. {quote}{{2018-08-24 04:57:28,685 | INFO | OMAudit | {color:#59afe1}CREATE_VOLUME{color} [{color:#654982}xiaoyu/172.18.0.3 {color}{color:#14892c}admin="xiaoyu" creationTime="0" owner="xiaoyu" quotaInBytes="1099511627776" volume="xmen"{color}] {color:#f79232}SUCCESS{color} |}} {quote} > I can definitely change the "username/IP" to "username IP" and ensure this > is at top level by logging it as ID. > Extensibility has been addressed by {color:#14892c}dataMap{color} {color:#14892c}> With this structure, the parsing will likely be easier as the actual data in enclosed in [ ]. Even if there were nested [ ] inside this, I feel it will still be easy to parse when we write a parser{color} > Given the above structure, unfortunately, we won't be able to log like the > example you provided Please do let me know if there are further improvements that can be made and thank you once again for a detailed feedback. :) was (Author: dineshchitlangia): [~jnp] Thank you for a detailed feedback. In this implementation, we are using the StructuredDataMessage (conforms to RFC 5424) for logging. Its basic structure is: {quote}{{{color:#59afe1}type{color} {color:#654982}ID{color} dataMap {color:#f79232}message{color} throwable}} {quote} * type - Customer action types we have defined for ozone like CREATE_VOLUME, CREATE_BUCKET etc * ID - I can log a string "username IPAddress" * dataMap - It is a key-value map object that we prepare for logging and can thus add as many key-value pairs as want. So this addresses the extensibility aspect. We cannot log username/IPAddress as key-value pair in the dataMap because the dataMap will sort by key while printing and currently there is no API to turn off sorting. * message - any specific message we may want to pass, it can also be left blank. We decided to pass this with only two values SUCCESS and FAILURE depending upon the result. In future when we build a log parser, this will be handy for querying various combinations. * throwable - if any exception was thrown in the actual operation, we can log it. This is totally configurable via properties file to log - x lines of exception or error message or not log at all. Thus, we end up with a log entry like the one below. {quote}{{2018-08-24 04:57:28,685 | INFO | OMAudit | {color:#59afe1}CREATE_VOLUME{color} [{color:#654982}xiaoyu/172.18.0.3 {color}{color:#14892c}admin="xiaoyu" creationTime="0" owner="xiaoyu" quotaInBytes="1099511627776" volume="xmen"{color}] {color:#f79232}SUCCESS{color} |}} {quote} > I can definitely change the "username/IP" to "username IP" and ensure this > is at top level by logging it as ID. > Extensibility has been addressed by {color:#14892c}dataMap{color} {color:#14892c}> With this structure, the parsing will likely be easier as the actual data in enclosed in [ ]. Even if there were nested [ ] inside this, I feel it will still be easy to parse when we write a parser{color} > Given the above structure, unfortunately, we won't be able to log like the > example you provided Please do let me know if there are further improvements that can be made and thank you once again for a detailed feedback. :) > Adding Ozone Manager Audit Log > ------------------------------ > > Key: HDDS-98 > URL: https://issues.apache.org/jira/browse/HDDS-98 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task > Reporter: Xiaoyu Yao > Assignee: Dinesh Chitlangia > Priority: Major > Labels: Logging, audit > Fix For: 0.2.1 > > Attachments: HDDS-98.001.patch, HDDS-98.002.patch, HDDS-98.003.patch, > HDDS-98.004.patch, audit.log, log4j2.properties > > > This ticket is opened to add ozone manager's audit log. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org