[
https://issues.apache.org/jira/browse/TS-2145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yunkai Zhang updated TS-2145:
-----------------------------
Description:
Without accurate stats, we don't known whether the log is lost.
This coming patch will be used to verify:
1) At log client side, how many logs genterated from HttpSM, how many logs sent
to LogServer.
2) At log server side, how many logs received from LogClient, how many logs
written to LogServers's disk.
==
I have attached a patch. Let's show an example for the metrics collected by
this patch:
In this example, there are two machines: one log-client and one log-server.
{code}
1) Clear old stats data in both client and server machines:
$ /etc/init.d/trafficserver stop
$ rm -rf /var/run/trafficserver/*.snap
2) Make logs_xml.config in log server empty.
3) Make logs_xml.config in log client looks like:
<LogObject>
<Format = "Cdn_access_log"/>
<CollationHosts = "<log-server-ip>:8085"/>
<Filename = "cdn_access"/>
<Protocols = "http"/>
<Filters = "only_get"/>
</LogObject>
2) Start log server (with collation mode == 1, logging_enable = 3,
squid_log_enabled = 0)
[log-server]$ /etc/init.d/trafficserver start
3) Start log client (with collation mode = 0, logging_enable = 3,
squid_log_enabled = 0)
[log-server]$ /etc/init.d/trafficserver start
4) Use jtest do some request for log client.
5) Stop log client and for a while (so that all data in network have been sent
to log-server).
6) Log client's metrics for logging:
[log-client]# lynx -dump http://localhost:8080/stat/ | grep process.log
proxy.process.log.event_log_error=0
proxy.process.log.event_log_error_skip=0
proxy.process.log.event_log_error_aggr=0
proxy.process.log.event_log_error_fail=0
proxy.process.log.event_log_access=4990275
proxy.process.log.event_log_access_skip=0
proxy.process.log.event_log_access_aggr=0
proxy.process.log.event_log_access_fail=0
proxy.process.log.num_sent_to_network=4990275
proxy.process.log.num_received_from_network=0
proxy.process.log.num_flush_to_disk=0
proxy.process.log.bytes_sent_to_network=719149632
proxy.process.log.bytes_received_from_network=0
proxy.process.log.bytes_flush_to_disk=0
proxy.process.log.bytes_written_to_disk=0
proxy.process.log.log_files_open=0
proxy.process.log.log_files_space_used=5330
7) Log server's metrics for logging:
[log-server]# lynx -dump http://localhost:8080/stat/ | grep process.log
proxy.process.log.event_log_error=0
proxy.process.log.event_log_error_skip=0
proxy.process.log.event_log_error_aggr=0
proxy.process.log.event_log_error_fail=0
proxy.process.log.event_log_access=0
proxy.process.log.event_log_access_skip=20
proxy.process.log.event_log_access_aggr=0
proxy.process.log.event_log_access_fail=0
proxy.process.log.num_sent_to_network=0
proxy.process.log.num_received_from_network=4990275
proxy.process.log.num_flush_to_disk=4990275
proxy.process.log.bytes_sent_to_network=0
proxy.process.log.bytes_received_from_network=719149632
proxy.process.log.bytes_flush_to_disk=485256906
proxy.process.log.bytes_written_to_disk=485256906
proxy.process.log.log_files_open=1
proxy.process.log.log_files_space_used=85215225070
{code}
was:
Without accurate stats, we don't known whether the log is lost.
This coming patch will be used to verify:
1) At log client side, how many logs genterated from HttpSM, how many logs sent
to LogServer.
2) At log server side, how many logs received from LogClient, how many logs
written to LogServers's disk.
> Add metrics for log collation, e.g. messages sent and messages received
> -----------------------------------------------------------------------
>
> Key: TS-2145
> URL: https://issues.apache.org/jira/browse/TS-2145
> Project: Traffic Server
> Issue Type: Improvement
> Components: Logging
> Reporter: Yunkai Zhang
> Assignee: Yunkai Zhang
> Attachments: 0001-TS-2145-Add-metrics-for-log-collation.patch
>
>
> Without accurate stats, we don't known whether the log is lost.
> This coming patch will be used to verify:
> 1) At log client side, how many logs genterated from HttpSM, how many logs
> sent to LogServer.
> 2) At log server side, how many logs received from LogClient, how many logs
> written to LogServers's disk.
> ==
> I have attached a patch. Let's show an example for the metrics collected by
> this patch:
> In this example, there are two machines: one log-client and one log-server.
> {code}
> 1) Clear old stats data in both client and server machines:
> $ /etc/init.d/trafficserver stop
> $ rm -rf /var/run/trafficserver/*.snap
> 2) Make logs_xml.config in log server empty.
> 3) Make logs_xml.config in log client looks like:
> <LogObject>
> <Format = "Cdn_access_log"/>
> <CollationHosts = "<log-server-ip>:8085"/>
> <Filename = "cdn_access"/>
> <Protocols = "http"/>
> <Filters = "only_get"/>
> </LogObject>
> 2) Start log server (with collation mode == 1, logging_enable = 3,
> squid_log_enabled = 0)
> [log-server]$ /etc/init.d/trafficserver start
> 3) Start log client (with collation mode = 0, logging_enable = 3,
> squid_log_enabled = 0)
> [log-server]$ /etc/init.d/trafficserver start
> 4) Use jtest do some request for log client.
> 5) Stop log client and for a while (so that all data in network have been
> sent to log-server).
> 6) Log client's metrics for logging:
> [log-client]# lynx -dump http://localhost:8080/stat/ | grep process.log
> proxy.process.log.event_log_error=0
> proxy.process.log.event_log_error_skip=0
> proxy.process.log.event_log_error_aggr=0
> proxy.process.log.event_log_error_fail=0
> proxy.process.log.event_log_access=4990275
> proxy.process.log.event_log_access_skip=0
> proxy.process.log.event_log_access_aggr=0
> proxy.process.log.event_log_access_fail=0
> proxy.process.log.num_sent_to_network=4990275
> proxy.process.log.num_received_from_network=0
> proxy.process.log.num_flush_to_disk=0
> proxy.process.log.bytes_sent_to_network=719149632
> proxy.process.log.bytes_received_from_network=0
> proxy.process.log.bytes_flush_to_disk=0
> proxy.process.log.bytes_written_to_disk=0
> proxy.process.log.log_files_open=0
> proxy.process.log.log_files_space_used=5330
> 7) Log server's metrics for logging:
> [log-server]# lynx -dump http://localhost:8080/stat/ | grep process.log
> proxy.process.log.event_log_error=0
> proxy.process.log.event_log_error_skip=0
> proxy.process.log.event_log_error_aggr=0
> proxy.process.log.event_log_error_fail=0
> proxy.process.log.event_log_access=0
> proxy.process.log.event_log_access_skip=20
> proxy.process.log.event_log_access_aggr=0
> proxy.process.log.event_log_access_fail=0
> proxy.process.log.num_sent_to_network=0
> proxy.process.log.num_received_from_network=4990275
> proxy.process.log.num_flush_to_disk=4990275
> proxy.process.log.bytes_sent_to_network=0
> proxy.process.log.bytes_received_from_network=719149632
> proxy.process.log.bytes_flush_to_disk=485256906
> proxy.process.log.bytes_written_to_disk=485256906
> proxy.process.log.log_files_open=1
> proxy.process.log.log_files_space_used=85215225070
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
