Re: [PATCH v1] qapi/qmp: Add timestamps to qmp command responses

2022-10-07 Thread Daniel P . Berrangé
On Fri, Oct 07, 2022 at 10:52:08AM +0300, Denis Plotnikov wrote:
> Add "start" & "end" time values to qmp command responses.
> 
> These time values are added to let the qemu management layer get the exact
> command execution time without any other time variance which might be brought 
> by
> other parts of management layer or qemu internals. This is particulary useful
> for the management layer logging for later problems resolving.
> 
> Example of result:
> 
> ./qemu/scripts/qmp/qmp-shell /tmp/qmp.socket
> 
> (QEMU) query-status
> {"end": {"seconds": 1650367305, "microseconds": 831032},
>  "start": {"seconds": 1650367305, "microseconds": 831012},
>  "return": {"status": "running", "singlestep": false, "running": true}}
> 
> The responce of the qmp command contains the start & end time of
> the qmp command processing.
> 
> Suggested-by: Andrey Ryabinin 
> Signed-off-by: Denis Plotnikov 
> ---
> v0->v1:
>  - remove interface to control "start" and "end" time values: return 
> timestamps unconditionally
>  - add description to qmp specification
>  - leave the same timestamp format in "seconds", "microseconds" to be 
> consistent with events
>timestamp
>  - fix patch description
> 
>  docs/interop/qmp-spec.txt | 20 ++--
>  qapi/qmp-dispatch.c   | 18 ++
>  2 files changed, 36 insertions(+), 2 deletions(-)
> 
> diff --git a/docs/interop/qmp-spec.txt b/docs/interop/qmp-spec.txt
> index b0e8351d5b261..d1cca8bc447ce 100644
> --- a/docs/interop/qmp-spec.txt
> +++ b/docs/interop/qmp-spec.txt
> @@ -158,7 +158,9 @@ responses that have an unknown "id" field.
>  
>  The format of a success response is:
>  
> -{ "return": json-value, "id": json-value }
> +{ "return": json-value, "id": json-value,
> +  "start": {"seconds": json-value, "microseconds": json-value},
> +  "end": {"seconds": json-value, "microseconds": json-value} }
>  
>   Where,
>  
> @@ -169,13 +171,21 @@ The format of a success response is:
>command does not return data
>  - The "id" member contains the transaction identification associated
>with the command execution if issued by the Client
> +- The "start" member contains the exact time of when the command has been
> +  stated to be processed. It is a fixed json-object with time in
> +  seconds and microseconds relative to the Unix Epoch (1 Jan 1970)

If I may make a suggestion

- The "start" member contains the exact time of when the server
  started executing the command. This excludes any time the
  command request spent queued, after reading it off the wire.

  It is a fixed json-object with time in seconds and microseconds
  relative to the Unix Epoch (1 Jan 1970)

> +- The "end" member contains the exact time of when the command has been
> +  finished to be processed. It is a fixed json-object with time in
> +  seconds and microseconds relative to the Unix Epoch (1 Jan 1970)

- The "end" member contains the exact time of when the server
  finished executing the command. This excludes any time the
  command response spent queued, waiting to be sent on the wire.

  It is a fixed json-object with time in seconds and microseconds
  relative to the Unix Epoch (1 Jan 1970)

> @@ -184,6 +194,12 @@ The format of an error response is:
>not attempt to parse this message.
>  - The "id" member contains the transaction identification associated with
>the command execution if issued by the Client
> +- The "start" member contains the exact time of when the command has been
> +  stated to be processed. It is a fixed json-object with time in
> +  seconds and microseconds relative to the Unix Epoch (1 Jan 1970)
> +- The "end" member contains the exact time of when the command has been
> +  finished to be processed. It is a fixed json-object with time in
> +  seconds and microseconds relative to the Unix Epoch (1 Jan 1970)

Same suggestion as above of course.


Could you also add something to tests/qtest/qmp-test.c to validate
that the start/end fields are present for command responses and
errors, and also validate that end > start as a sanity check.


IIUC, this change will transparently apply to the QEMU guest agent
too since your change looks to be in the shared code for QMP ?

With regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|




[PATCH v1] qapi/qmp: Add timestamps to qmp command responses

2022-10-07 Thread Denis Plotnikov
Add "start" & "end" time values to qmp command responses.

These time values are added to let the qemu management layer get the exact
command execution time without any other time variance which might be brought by
other parts of management layer or qemu internals. This is particulary useful
for the management layer logging for later problems resolving.

Example of result:

./qemu/scripts/qmp/qmp-shell /tmp/qmp.socket

(QEMU) query-status
{"end": {"seconds": 1650367305, "microseconds": 831032},
 "start": {"seconds": 1650367305, "microseconds": 831012},
 "return": {"status": "running", "singlestep": false, "running": true}}

The responce of the qmp command contains the start & end time of
the qmp command processing.

Suggested-by: Andrey Ryabinin 
Signed-off-by: Denis Plotnikov 
---
v0->v1:
 - remove interface to control "start" and "end" time values: return timestamps 
unconditionally
 - add description to qmp specification
 - leave the same timestamp format in "seconds", "microseconds" to be 
consistent with events
   timestamp
 - fix patch description

 docs/interop/qmp-spec.txt | 20 ++--
 qapi/qmp-dispatch.c   | 18 ++
 2 files changed, 36 insertions(+), 2 deletions(-)

diff --git a/docs/interop/qmp-spec.txt b/docs/interop/qmp-spec.txt
index b0e8351d5b261..d1cca8bc447ce 100644
--- a/docs/interop/qmp-spec.txt
+++ b/docs/interop/qmp-spec.txt
@@ -158,7 +158,9 @@ responses that have an unknown "id" field.
 
 The format of a success response is:
 
-{ "return": json-value, "id": json-value }
+{ "return": json-value, "id": json-value,
+  "start": {"seconds": json-value, "microseconds": json-value},
+  "end": {"seconds": json-value, "microseconds": json-value} }
 
  Where,
 
@@ -169,13 +171,21 @@ The format of a success response is:
   command does not return data
 - The "id" member contains the transaction identification associated
   with the command execution if issued by the Client
+- The "start" member contains the exact time of when the command has been
+  stated to be processed. It is a fixed json-object with time in
+  seconds and microseconds relative to the Unix Epoch (1 Jan 1970)
+- The "end" member contains the exact time of when the command has been
+  finished to be processed. It is a fixed json-object with time in
+  seconds and microseconds relative to the Unix Epoch (1 Jan 1970)
 
 2.4.2 error
 ---
 
 The format of an error response is:
 
-{ "error": { "class": json-string, "desc": json-string }, "id": json-value }
+{ "error": { "class": json-string, "desc": json-string }, "id": json-value
+  "start": {"seconds": json-value, "microseconds": json-value},
+  "end": {"seconds": json-value, "microseconds": json-value} }
 
  Where,
 
@@ -184,6 +194,12 @@ The format of an error response is:
   not attempt to parse this message.
 - The "id" member contains the transaction identification associated with
   the command execution if issued by the Client
+- The "start" member contains the exact time of when the command has been
+  stated to be processed. It is a fixed json-object with time in
+  seconds and microseconds relative to the Unix Epoch (1 Jan 1970)
+- The "end" member contains the exact time of when the command has been
+  finished to be processed. It is a fixed json-object with time in
+  seconds and microseconds relative to the Unix Epoch (1 Jan 1970)
 
 NOTE: Some errors can occur before the Server is able to read the "id" member,
 in these cases the "id" member will not be part of the error response, even
diff --git a/qapi/qmp-dispatch.c b/qapi/qmp-dispatch.c
index 0990873ec8ec1..fce87416f2128 100644
--- a/qapi/qmp-dispatch.c
+++ b/qapi/qmp-dispatch.c
@@ -130,6 +130,22 @@ static void do_qmp_dispatch_bh(void *opaque)
 aio_co_wake(data->co);
 }
 
+static void add_timestamps(QDict *qdict, uint64_t start_ms, uint64_t end_ms)
+{
+QDict *start_dict, *end_dict;
+
+start_dict = qdict_new();
+qdict_put_int(start_dict, "seconds", start_ms / G_USEC_PER_SEC);
+qdict_put_int(start_dict, "microseconds", start_ms % G_USEC_PER_SEC);
+
+end_dict = qdict_new();
+qdict_put_int(end_dict, "seconds", end_ms / G_USEC_PER_SEC);
+qdict_put_int(end_dict, "microseconds", end_ms % G_USEC_PER_SEC);
+
+qdict_put_obj(qdict, "start", QOBJECT(start_dict));
+qdict_put_obj(qdict, "end", QOBJECT(end_dict));
+}
+
 /*
  * Runs outside of coroutine context for OOB commands, but in coroutine
  * context for everything else.
@@ -146,6 +162,7 @@ QDict *qmp_dispatch(const QmpCommandList *cmds, QObject 
*request,
 QObject *id;
 QObject *ret = NULL;
 QDict *rsp = NULL;
+uint64_t ts_start = g_get_real_time();
 
 dict = qobject_to(QDict, request);
 if (!dict) {
@@ -270,5 +287,6 @@ out:
 qdict_put_obj(rsp, "id", qobject_ref(id));
 }
 
+add_timestamps(rsp, ts_start, g_get_real_time());
 return rsp;
 }
-- 
2.25.1