[
https://issues.apache.org/jira/browse/TS-2145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13753603#comment-13753603
]
ASF subversion and git services commented on TS-2145:
-----------------------------------------------------
Commit 9d2acd0a524d7d74cc41dc280c28a67b9eb72405 in branch refs/heads/master
from [~yunkai]
[ https://git-wip-us.apache.org/repos/asf?p=trafficserver.git;h=9d2acd0 ]
TS-2145: Add metrics for log collation
Without accurate stats, we don't known whether the log is lost.
This patch is used to verify, such as:
1) At log client side, how many logs genterated from HttpSM, how many
logs sent to LogServer.
2) At log server side, how many logs received from LogClient, how many
logs written to LogServers's disk.
After apply this patch, we will get the following metrics for logging:
a) Events of Log::error():
proxy.process.log.event_log_error_ok
proxy.process.log.event_log_error_skip
proxy.process.log.event_log_error_aggr
proxy.process.log.event_log_error_full
proxy.process.log.event_log_error_fail
b) Events for Log::access():
proxy.process.log.event_log_access_ok
proxy.process.log.event_log_access_skip
proxy.process.log.event_log_access_aggr
proxy.process.log.event_log_access_full
proxy.process.log.event_log_access_fail
c) Logging Data to/from network/disk:
proxy.process.log.num_sent_to_network
proxy.process.log.num_received_from_network
proxy.process.log.num_flush_to_disk
proxy.process.log.bytes_sent_to_network
proxy.process.log.bytes_received_from_network
proxy.process.log.bytes_flush_to_disk
proxy.process.log.bytes_written_to_disk
d) Mix:
proxy.process.log.log_files_open
proxy.process.log.log_files_space_used
BTW: Remove LOG_XXX stats macro according Leif's suggestion.
Signed-off-by: Yunkai Zhang <[email protected]>
> Add metrics for log collation, e.g. messages sent and messages received
> -----------------------------------------------------------------------
>
> Key: TS-2145
> URL: https://issues.apache.org/jira/browse/TS-2145
> Project: Traffic Server
> Issue Type: Improvement
> Components: Logging
> Reporter: Yunkai Zhang
> Assignee: Yunkai Zhang
> Attachments: 0001-TS-2145-Add-metrics-for-log-collation.patch,
> 0001-TS-2145-Add-metrics-for-log-collation.V2.patch
>
>
> Without accurate stats, we don't known whether the log is lost.
> This coming patch will be used to verify:
> 1) At log client side, how many logs genterated from HttpSM, how many logs
> sent to LogServer.
> 2) At log server side, how many logs received from LogClient, how many logs
> written to LogServers's disk.
> ==
> I have attached a patch. Let's show an example for the metrics collected by
> this patch:
> In this example, there are two machines: one log-client and one log-server.
> {code}
> 1) Clear old stats data in both client and server machines:
> $ /etc/init.d/trafficserver stop
> $ rm -rf /var/run/trafficserver/*.snap
> 2) Make logs_xml.config in log server empty.
> 3) Make logs_xml.config in log client looks like:
> <LogObject>
> <Format = "Cdn_access_log"/>
> <CollationHosts = "<log-server-ip>:8085"/>
> <Filename = "cdn_access"/>
> <Protocols = "http"/>
> <Filters = "only_get"/>
> </LogObject>
> 2) Start log server (with collation mode == 1, logging_enable = 3,
> squid_log_enabled = 0)
> [log-server]$ /etc/init.d/trafficserver start
> 3) Start log client (with collation mode = 0, logging_enable = 3,
> squid_log_enabled = 0)
> [log-server]$ /etc/init.d/trafficserver start
> 4) Use jtest do some request for log client.
> 5) Stop log client and for a while (so that all data in network have been
> sent to log-server).
> 6) Log client's metrics for logging:
> [log-client]# lynx -dump http://localhost:8080/stat/ | grep process.log
> proxy.process.log.event_log_error=0
> proxy.process.log.event_log_error_skip=0
> proxy.process.log.event_log_error_aggr=0
> proxy.process.log.event_log_error_fail=0
> proxy.process.log.event_log_access=4990275
> proxy.process.log.event_log_access_skip=0
> proxy.process.log.event_log_access_aggr=0
> proxy.process.log.event_log_access_fail=0
> proxy.process.log.num_sent_to_network=4990275
> proxy.process.log.num_received_from_network=0
> proxy.process.log.num_flush_to_disk=0
> proxy.process.log.bytes_sent_to_network=719149632
> proxy.process.log.bytes_received_from_network=0
> proxy.process.log.bytes_flush_to_disk=0
> proxy.process.log.bytes_written_to_disk=0
> proxy.process.log.log_files_open=0
> proxy.process.log.log_files_space_used=5330
> 7) Log server's metrics for logging:
> [log-server]# lynx -dump http://localhost:8080/stat/ | grep process.log
> proxy.process.log.event_log_error=0
> proxy.process.log.event_log_error_skip=0
> proxy.process.log.event_log_error_aggr=0
> proxy.process.log.event_log_error_fail=0
> proxy.process.log.event_log_access=0
> proxy.process.log.event_log_access_skip=20
> proxy.process.log.event_log_access_aggr=0
> proxy.process.log.event_log_access_fail=0
> proxy.process.log.num_sent_to_network=0
> proxy.process.log.num_received_from_network=4990275
> proxy.process.log.num_flush_to_disk=4990275
> proxy.process.log.bytes_sent_to_network=0
> proxy.process.log.bytes_received_from_network=719149632
> proxy.process.log.bytes_flush_to_disk=485256906
> proxy.process.log.bytes_written_to_disk=485256906
> proxy.process.log.log_files_open=1
> proxy.process.log.log_files_space_used=85215225070
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
