> DPDK could produce huge amount of logs. For example, in case of exhausting
> of a mempool in vhost-user port, following message will be printed on each
> call to 'rte_vhost_dequeue_burst()':
>
> |ERR|VHOST_DATA: Failed to allocate memory for mbuf.
>
> These messages are increasing ovs-vswitchd.log size extremely fast making
> it unreadable and non-parsable by a common linux utils like grep, less
> etc. Moreover continuously growing log could exhaust the HDD space in a
> few hours breaking normal operation of the whole system.
>
> To avoid such issues, DPDK log rate limited to 600 messages per minute.
> This value is high, because we still want to see many big logs like vhost-
> user configuration sequence. The debug messages are treated separately to
> avoid looss of errors/warnings in case of intensive debug enabled in DPDK.
>
> Signed-off-by: Ilya Maximets <[email protected]>
Thanks for this Ilya, LGTM.
Do you think this is worth backporting to previous releases also, the same
issue would be there also I assume?
Ian
> ---
>
> Version 2:
> * Separate limit for DEBUG level.
>
> lib/dpdk.c | 10 ++++++----
> 1 file changed, 6 insertions(+), 4 deletions(-)
>
> diff --git a/lib/dpdk.c b/lib/dpdk.c
> index 3f5a55f..00dd974 100644
> --- a/lib/dpdk.c
> +++ b/lib/dpdk.c
> @@ -272,20 +272,22 @@ static ssize_t
> dpdk_log_write(void *c OVS_UNUSED, const char *buf, size_t size) {
> char *str = xmemdup0(buf, size);
> + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(600, 600);
> + static struct vlog_rate_limit dbg_rl = VLOG_RATE_LIMIT_INIT(600,
> + 600);
>
> switch (rte_log_cur_msg_loglevel()) {
> case RTE_LOG_DEBUG:
> - VLOG_DBG("%s", str);
> + VLOG_DBG_RL(&dbg_rl, "%s", str);
> break;
> case RTE_LOG_INFO:
> case RTE_LOG_NOTICE:
> - VLOG_INFO("%s", str);
> + VLOG_INFO_RL(&rl, "%s", str);
> break;
> case RTE_LOG_WARNING:
> - VLOG_WARN("%s", str);
> + VLOG_WARN_RL(&rl, "%s", str);
> break;
> case RTE_LOG_ERR:
> - VLOG_ERR("%s", str);
> + VLOG_ERR_RL(&rl, "%s", str);
> break;
> case RTE_LOG_CRIT:
> case RTE_LOG_ALERT:
> --
> 2.7.4
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
