Re: [Qemu-devel] [PATCH 02/15] colo-compare: implement the process of checkpoint

2017-04-21 Thread Hailiang Zhang 

On 2017/4/20 13:15, Jason Wang wrote:


On 2017年04月18日 14:58, Hailiang Zhang wrote:

On 2017/4/18 11:55, Jason Wang wrote:

On 2017年04月17日 19:04, Hailiang Zhang wrote:

Hi Jason,

On 2017/4/14 14:38, Jason Wang wrote:

On 2017年04月14日 14:22, Hailiang Zhang wrote:

Hi Jason,

On 2017/4/14 13:57, Jason Wang wrote:

On 2017年02月22日 17:31, Zhang Chen wrote:

On 02/22/2017 11:42 AM, zhanghailiang wrote:

While do checkpoint, we need to flush all the unhandled packets,
By using the filter notifier mechanism, we can easily to notify
every compare object to do this process, which runs inside
of compare threads as a coroutine.

Hi~ Jason and Hailiang.

I will send a patch set later about colo-compare notify
mechanism for
Xen like this patch.
I want to add a new chardev socket way in colo-comapre connect
to Xen
colo, for notify
checkpoint or failover, Because We have no choice to use this way
communicate with Xen codes.
That's means we will have two notify mechanism.
What do you think about this?


Thanks
Zhang Chen

I was thinking the possibility of using similar way to for colo
compare.
E.g can we use socket? This can saves duplicated codes more or less.

Since there are too many sockets used by filter and COLO, (Two unix
sockets and two
tcp sockets for each vNIC), I don't want to introduce more ;) ,
but
i'm not sure if it is
possible to make it more flexible and optional, abstract these
duplicated codes,
pass the opened fd (No matter eventfd or socket fd ) as parameter,
for
example.
Is this way acceptable ?

Thanks,
Hailiang

Yes, that's kind of what I want. We don't want to use two message
format. Passing a opened fd need management support, we still need a
fallback if there's no management on top. For qemu/kvm, we can do all
stuffs transparent to the cli by e.g socketpair() or others, but
the key
is to have a unified message format.

After a deeper investigation, i think we can re-use most codes, since
there is no
existing way to notify xen (no ?), we still needs notify chardev
socket (Be used to notify xen, it is optional.)
(http://patchwork.ozlabs.org/patch/733431/ "COLO-compare: Add Xen
notify chardev socket handler frame")

Yes and actually you can use this for bi-directional communication. The
only differences is the implementation of comparing.


Besides, there is an existing qmp comand 'xen-colo-do-checkpoint',

I don't see this in master?

Er, it has been merged already, please see migration/colo.c, void
qmp_xen_colo_do_checkpoint(Error **errp);

Aha, I see. Thanks.


;)


we can re-use it to notify
colo-compare objects and other filter objects to do checkpoint, for
the opposite direction, we use
the notify chardev socket (Only for xen).

Just want to make sure I understand the design, who will trigger this
command? Management?

The command will be issued by XEN (xc_save ?), the original existing
xen-colo-do-checkpoint
command now only be used to notify block replication to do checkpoint,
we can re-use it for filters too.

So it was called by management. For KVM case, we probably don't need
this since the comparing thread are under control of qemu.


Yes, you are right.


Can we just use the socket?

I don't quite understand ...
Just as the codes showed bellow, in this scenario,
XEN notifies colo-compare and fiters do checkpoint by using qmp command,

Yes, that's just what I mean. Technically Xen can use socket to do this too.


Yes, great, since we have come to an agreement on the scenario, I'll update 
this series.

Thanks,
Hailiang.


Thanks


and colo-compare notifies XEN about net inconsistency event by using
the new socket.


So the codes will be like:
diff --git a/migration/colo.c b/migration/colo.c
index 91da936..813c281 100644
--- a/migration/colo.c
+++ b/migration/colo.c
@@ -224,7 +224,19 @@ ReplicationStatus
*qmp_query_xen_replication_status(Error **errp)

   void qmp_xen_colo_do_checkpoint(Error **errp)
   {
+Error *local_err = NULL;
+
   replication_do_checkpoint_all(errp);
+/* Notify colo-compare and other filters to do checkpoint */
+colo_notify_compares_event(NULL, COLO_CHECKPOINT, _err);
+if (local_err) {
+error_propagate(errp, local_err);
+return;
+}
+colo_notify_filters_event(COLO_CHECKPOINT, _err);
+if (local_err) {
+error_propagate(errp, local_err);
+}
   }

   static void colo_send_message(QEMUFile *f, COLOMessage msg,
diff --git a/net/colo-compare.c b/net/colo-compare.c
index 24e13f0..de975c5 100644
--- a/net/colo-compare.c
+++ b/net/colo-compare.c
@@ -391,6 +391,9 @@ static void colo_compare_inconsistent_notify(void)
   {
   notifier_list_notify(_compare_notifiers,
   migrate_get_current());

KVM will use this notifier/callback way, and in this way, we can avoid
the redundant socket.


+if (s->notify_dev) {
+   /* Do something, notify remote side through notify dev */
+}
   }

If we have a notify socket configured, we will send the message about
net inconsistent event.


   void

Re: [Qemu-devel] [PATCH 02/15] colo-compare: implement the process of checkpoint

2017-04-19 Thread Jason Wang 



On 2017年04月18日 14:58, Hailiang Zhang wrote:

On 2017/4/18 11:55, Jason Wang wrote:


On 2017年04月17日 19:04, Hailiang Zhang wrote:

Hi Jason,

On 2017/4/14 14:38, Jason Wang wrote:

On 2017年04月14日 14:22, Hailiang Zhang wrote:

Hi Jason,

On 2017/4/14 13:57, Jason Wang wrote:

On 2017年02月22日 17:31, Zhang Chen wrote:

On 02/22/2017 11:42 AM, zhanghailiang wrote:

While do checkpoint, we need to flush all the unhandled packets,
By using the filter notifier mechanism, we can easily to notify
every compare object to do this process, which runs inside
of compare threads as a coroutine.

Hi~ Jason and Hailiang.

I will send a patch set later about colo-compare notify 
mechanism for

Xen like this patch.
I want to add a new chardev socket way in colo-comapre connect 
to Xen

colo, for notify
checkpoint or failover, Because We have no choice to use this way
communicate with Xen codes.
That's means we will have two notify mechanism.
What do you think about this?


Thanks
Zhang Chen

I was thinking the possibility of using similar way to for colo
compare.
E.g can we use socket? This can saves duplicated codes more or less.

Since there are too many sockets used by filter and COLO, (Two unix
sockets and two
   tcp sockets for each vNIC), I don't want to introduce more ;) , 
but

i'm not sure if it is
possible to make it more flexible and optional, abstract these
duplicated codes,
pass the opened fd (No matter eventfd or socket fd ) as parameter, 
for

example.
Is this way acceptable ?

Thanks,
Hailiang

Yes, that's kind of what I want. We don't want to use two message
format. Passing a opened fd need management support, we still need a
fallback if there's no management on top. For qemu/kvm, we can do all
stuffs transparent to the cli by e.g socketpair() or others, but 
the key

is to have a unified message format.

After a deeper investigation, i think we can re-use most codes, since
there is no
existing way to notify xen (no ?), we still needs notify chardev
socket (Be used to notify xen, it is optional.)
(http://patchwork.ozlabs.org/patch/733431/ "COLO-compare: Add Xen
notify chardev socket handler frame")

Yes and actually you can use this for bi-directional communication. The
only differences is the implementation of comparing.


Besides, there is an existing qmp comand 'xen-colo-do-checkpoint',

I don't see this in master?


Er, it has been merged already, please see migration/colo.c, void 
qmp_xen_colo_do_checkpoint(Error **errp);


Aha, I see. Thanks.




we can re-use it to notify
colo-compare objects and other filter objects to do checkpoint, for
the opposite direction, we use
the notify chardev socket (Only for xen).

Just want to make sure I understand the design, who will trigger this
command? Management?


The command will be issued by XEN (xc_save ?), the original existing 
xen-colo-do-checkpoint
command now only be used to notify block replication to do checkpoint, 
we can re-use it for filters too.


So it was called by management. For KVM case, we probably don't need 
this since the comparing thread are under control of qemu.





Can we just use the socket?


I don't quite understand ...
Just as the codes showed bellow, in this scenario,
XEN notifies colo-compare and fiters do checkpoint by using qmp command,


Yes, that's just what I mean. Technically Xen can use socket to do this too.

Thanks

and colo-compare notifies XEN about net inconsistency event by using 
the new socket.



So the codes will be like:
diff --git a/migration/colo.c b/migration/colo.c
index 91da936..813c281 100644
--- a/migration/colo.c
+++ b/migration/colo.c
@@ -224,7 +224,19 @@ ReplicationStatus
*qmp_query_xen_replication_status(Error **errp)

  void qmp_xen_colo_do_checkpoint(Error **errp)
  {
+Error *local_err = NULL;
+
  replication_do_checkpoint_all(errp);
+/* Notify colo-compare and other filters to do checkpoint */
+colo_notify_compares_event(NULL, COLO_CHECKPOINT, _err);
+if (local_err) {
+error_propagate(errp, local_err);
+return;
+}
+colo_notify_filters_event(COLO_CHECKPOINT, _err);
+if (local_err) {
+error_propagate(errp, local_err);
+}
  }

  static void colo_send_message(QEMUFile *f, COLOMessage msg,
diff --git a/net/colo-compare.c b/net/colo-compare.c
index 24e13f0..de975c5 100644
--- a/net/colo-compare.c
+++ b/net/colo-compare.c
@@ -391,6 +391,9 @@ static void colo_compare_inconsistent_notify(void)
  {
  notifier_list_notify(_compare_notifiers,
  migrate_get_current());


KVM will use this notifier/callback way, and in this way, we can avoid 
the redundant socket.



+if (s->notify_dev) {
+   /* Do something, notify remote side through notify dev */
+}
  }


If we have a notify socket configured, we will send the message about 
net inconsistent event.




  void colo_compare_register_notifier(Notifier *notify)

How about this scenario ？

See my reply above, and we need unify the message format too. Raw string
is ok but

Re: [Qemu-devel] [PATCH 02/15] colo-compare: implement the process of checkpoint

2017-04-18 Thread Hailiang Zhang 

On 2017/4/18 11:55, Jason Wang wrote:


On 2017年04月17日 19:04, Hailiang Zhang wrote:

Hi Jason,

On 2017/4/14 14:38, Jason Wang wrote:

On 2017年04月14日 14:22, Hailiang Zhang wrote:

Hi Jason,

On 2017/4/14 13:57, Jason Wang wrote:

On 2017年02月22日 17:31, Zhang Chen wrote:

On 02/22/2017 11:42 AM, zhanghailiang wrote:

While do checkpoint, we need to flush all the unhandled packets,
By using the filter notifier mechanism, we can easily to notify
every compare object to do this process, which runs inside
of compare threads as a coroutine.

Hi~ Jason and Hailiang.

I will send a patch set later about colo-compare notify mechanism for
Xen like this patch.
I want to add a new chardev socket way in colo-comapre connect to Xen
colo, for notify
checkpoint or failover, Because We have no choice to use this way
communicate with Xen codes.
That's means we will have two notify mechanism.
What do you think about this?


Thanks
Zhang Chen

I was thinking the possibility of using similar way to for colo
compare.
E.g can we use socket? This can saves duplicated codes more or less.

Since there are too many sockets used by filter and COLO, (Two unix
sockets and two
   tcp sockets for each vNIC), I don't want to introduce more ;) , but
i'm not sure if it is
possible to make it more flexible and optional, abstract these
duplicated codes,
pass the opened fd (No matter eventfd or socket fd ) as parameter, for
example.
Is this way acceptable ?

Thanks,
Hailiang

Yes, that's kind of what I want. We don't want to use two message
format. Passing a opened fd need management support, we still need a
fallback if there's no management on top. For qemu/kvm, we can do all
stuffs transparent to the cli by e.g socketpair() or others, but the key
is to have a unified message format.

After a deeper investigation, i think we can re-use most codes, since
there is no
existing way to notify xen (no ?), we still needs notify chardev
socket (Be used to notify xen, it is optional.)
(http://patchwork.ozlabs.org/patch/733431/ "COLO-compare: Add Xen
notify chardev socket handler frame")

Yes and actually you can use this for bi-directional communication. The
only differences is the implementation of comparing.


Besides, there is an existing qmp comand 'xen-colo-do-checkpoint',

I don't see this in master?


Er, it has been merged already, please see migration/colo.c, void 
qmp_xen_colo_do_checkpoint(Error **errp);


we can re-use it to notify
colo-compare objects and other filter objects to do checkpoint, for
the opposite direction, we use
the notify chardev socket (Only for xen).

Just want to make sure I understand the design, who will trigger this
command? Management?


The command will be issued by XEN (xc_save ?), the original existing 
xen-colo-do-checkpoint
command now only be used to notify block replication to do checkpoint, we can 
re-use it for filters too.


Can we just use the socket?


I don't quite understand ...
Just as the codes showed bellow, in this scenario,
XEN notifies colo-compare and fiters do checkpoint by using qmp command,
and colo-compare notifies XEN about net inconsistency event by using the new 
socket.


So the codes will be like:
diff --git a/migration/colo.c b/migration/colo.c
index 91da936..813c281 100644
--- a/migration/colo.c
+++ b/migration/colo.c
@@ -224,7 +224,19 @@ ReplicationStatus
*qmp_query_xen_replication_status(Error **errp)

  void qmp_xen_colo_do_checkpoint(Error **errp)
  {
+Error *local_err = NULL;
+
  replication_do_checkpoint_all(errp);
+/* Notify colo-compare and other filters to do checkpoint */
+colo_notify_compares_event(NULL, COLO_CHECKPOINT, _err);
+if (local_err) {
+error_propagate(errp, local_err);
+return;
+}
+colo_notify_filters_event(COLO_CHECKPOINT, _err);
+if (local_err) {
+error_propagate(errp, local_err);
+}
  }

  static void colo_send_message(QEMUFile *f, COLOMessage msg,
diff --git a/net/colo-compare.c b/net/colo-compare.c
index 24e13f0..de975c5 100644
--- a/net/colo-compare.c
+++ b/net/colo-compare.c
@@ -391,6 +391,9 @@ static void colo_compare_inconsistent_notify(void)
  {
  notifier_list_notify(_compare_notifiers,
  migrate_get_current());


KVM will use this notifier/callback way, and in this way, we can avoid the 
redundant socket.


+if (s->notify_dev) {
+   /* Do something, notify remote side through notify dev */
+}
  }


If we have a notify socket configured, we will send the message about net 
inconsistent event.



  void colo_compare_register_notifier(Notifier *notify)

How about this scenario ？

See my reply above, and we need unify the message format too. Raw string
is ok but we'd better have something like TLV or others.


Agreed, we need it to be more standard.

Thanks,
Hailiang


Thanks


Thoughts?

Thanks


Thanks


.


.






.

Re: [Qemu-devel] [PATCH 02/15] colo-compare: implement the process of checkpoint

2017-04-17 Thread Jason Wang 



On 2017年04月17日 19:04, Hailiang Zhang wrote:

Hi Jason,

On 2017/4/14 14:38, Jason Wang wrote:


On 2017年04月14日 14:22, Hailiang Zhang wrote:

Hi Jason,

On 2017/4/14 13:57, Jason Wang wrote:

On 2017年02月22日 17:31, Zhang Chen wrote:

On 02/22/2017 11:42 AM, zhanghailiang wrote:

While do checkpoint, we need to flush all the unhandled packets,
By using the filter notifier mechanism, we can easily to notify
every compare object to do this process, which runs inside
of compare threads as a coroutine.

Hi~ Jason and Hailiang.

I will send a patch set later about colo-compare notify mechanism for
Xen like this patch.
I want to add a new chardev socket way in colo-comapre connect to Xen
colo, for notify
checkpoint or failover, Because We have no choice to use this way
communicate with Xen codes.
That's means we will have two notify mechanism.
What do you think about this?


Thanks
Zhang Chen
I was thinking the possibility of using similar way to for colo 
compare.

E.g can we use socket? This can saves duplicated codes more or less.

Since there are too many sockets used by filter and COLO, (Two unix
sockets and two
  tcp sockets for each vNIC), I don't want to introduce more ;) , but
i'm not sure if it is
possible to make it more flexible and optional, abstract these
duplicated codes,
pass the opened fd (No matter eventfd or socket fd ) as parameter, for
example.
Is this way acceptable ?

Thanks,
Hailiang

Yes, that's kind of what I want. We don't want to use two message
format. Passing a opened fd need management support, we still need a
fallback if there's no management on top. For qemu/kvm, we can do all
stuffs transparent to the cli by e.g socketpair() or others, but the key
is to have a unified message format.


After a deeper investigation, i think we can re-use most codes, since 
there is no
existing way to notify xen (no ?), we still needs notify chardev 
socket (Be used to notify xen, it is optional.)
(http://patchwork.ozlabs.org/patch/733431/ "COLO-compare: Add Xen 
notify chardev socket handler frame")


Yes and actually you can use this for bi-directional communication. The 
only differences is the implementation of comparing.




Besides, there is an existing qmp comand 'xen-colo-do-checkpoint', 


I don't see this in master?


we can re-use it to notify
colo-compare objects and other filter objects to do checkpoint, for 
the opposite direction, we use

the notify chardev socket (Only for xen).


Just want to make sure I understand the design, who will trigger this 
command? Management?


Can we just use the socket?



So the codes will be like:
diff --git a/migration/colo.c b/migration/colo.c
index 91da936..813c281 100644
--- a/migration/colo.c
+++ b/migration/colo.c
@@ -224,7 +224,19 @@ ReplicationStatus 
*qmp_query_xen_replication_status(Error **errp)


 void qmp_xen_colo_do_checkpoint(Error **errp)
 {
+Error *local_err = NULL;
+
 replication_do_checkpoint_all(errp);
+/* Notify colo-compare and other filters to do checkpoint */
+colo_notify_compares_event(NULL, COLO_CHECKPOINT, _err);
+if (local_err) {
+error_propagate(errp, local_err);
+return;
+}
+colo_notify_filters_event(COLO_CHECKPOINT, _err);
+if (local_err) {
+error_propagate(errp, local_err);
+}
 }

 static void colo_send_message(QEMUFile *f, COLOMessage msg,
diff --git a/net/colo-compare.c b/net/colo-compare.c
index 24e13f0..de975c5 100644
--- a/net/colo-compare.c
+++ b/net/colo-compare.c
@@ -391,6 +391,9 @@ static void colo_compare_inconsistent_notify(void)
 {
 notifier_list_notify(_compare_notifiers,
 migrate_get_current());
+if (s->notify_dev) {
+   /* Do something, notify remote side through notify dev */
+}
 }

 void colo_compare_register_notifier(Notifier *notify)

How about this scenario ？


See my reply above, and we need unify the message format too. Raw string 
is ok but we'd better have something like TLV or others.


Thanks




Thoughts?

Thanks


Thanks


.





.

Re: [Qemu-devel] [PATCH 02/15] colo-compare: implement the process of checkpoint

2017-04-17 Thread Zhang Chen 



On 04/17/2017 07:04 PM, Hailiang Zhang wrote:

Hi Jason,

On 2017/4/14 14:38, Jason Wang wrote:


On 2017年04月14日 14:22, Hailiang Zhang wrote:

Hi Jason,

On 2017/4/14 13:57, Jason Wang wrote:

On 2017年02月22日 17:31, Zhang Chen wrote:

On 02/22/2017 11:42 AM, zhanghailiang wrote:

While do checkpoint, we need to flush all the unhandled packets,
By using the filter notifier mechanism, we can easily to notify
every compare object to do this process, which runs inside
of compare threads as a coroutine.

Hi~ Jason and Hailiang.

I will send a patch set later about colo-compare notify mechanism for
Xen like this patch.
I want to add a new chardev socket way in colo-comapre connect to Xen
colo, for notify
checkpoint or failover, Because We have no choice to use this way
communicate with Xen codes.
That's means we will have two notify mechanism.
What do you think about this?


Thanks
Zhang Chen
I was thinking the possibility of using similar way to for colo 
compare.

E.g can we use socket? This can saves duplicated codes more or less.

Since there are too many sockets used by filter and COLO, (Two unix
sockets and two
  tcp sockets for each vNIC), I don't want to introduce more ;) , but
i'm not sure if it is
possible to make it more flexible and optional, abstract these
duplicated codes,
pass the opened fd (No matter eventfd or socket fd ) as parameter, for
example.
Is this way acceptable ?

Thanks,
Hailiang

Yes, that's kind of what I want. We don't want to use two message
format. Passing a opened fd need management support, we still need a
fallback if there's no management on top. For qemu/kvm, we can do all
stuffs transparent to the cli by e.g socketpair() or others, but the key
is to have a unified message format.


After a deeper investigation, i think we can re-use most codes, since 
there is no
existing way to notify xen (no ?), we still needs notify chardev 
socket (Be used to notify xen, it is optional.)
(http://patchwork.ozlabs.org/patch/733431/ "COLO-compare: Add Xen 
notify chardev socket handler frame")


Besides, there is an existing qmp comand 'xen-colo-do-checkpoint', we 
can re-use it to notify
colo-compare objects and other filter objects to do checkpoint, for 
the opposite direction, we use

the notify chardev socket (Only for xen).

So the codes will be like:
diff --git a/migration/colo.c b/migration/colo.c
index 91da936..813c281 100644
--- a/migration/colo.c
+++ b/migration/colo.c
@@ -224,7 +224,19 @@ ReplicationStatus 
*qmp_query_xen_replication_status(Error **errp)


 void qmp_xen_colo_do_checkpoint(Error **errp)
 {
+Error *local_err = NULL;
+
 replication_do_checkpoint_all(errp);
+/* Notify colo-compare and other filters to do checkpoint */
+colo_notify_compares_event(NULL, COLO_CHECKPOINT, _err);
+if (local_err) {
+error_propagate(errp, local_err);
+return;
+}
+colo_notify_filters_event(COLO_CHECKPOINT, _err);
+if (local_err) {
+error_propagate(errp, local_err);
+}
 }

 static void colo_send_message(QEMUFile *f, COLOMessage msg,
diff --git a/net/colo-compare.c b/net/colo-compare.c
index 24e13f0..de975c5 100644
--- a/net/colo-compare.c
+++ b/net/colo-compare.c
@@ -391,6 +391,9 @@ static void colo_compare_inconsistent_notify(void)
 {
 notifier_list_notify(_compare_notifiers,
 migrate_get_current());
+if (s->notify_dev) {
+   /* Do something, notify remote side through notify dev */
+}
 }

 void colo_compare_register_notifier(Notifier *notify)

How about this scenario ？


I agree this way, maybe rename qmp_xen_colo_do_checkpoint()
to qmp_remote_colo_do_checkpoint() is more generic.

Thanks
Zhang Chen




Thoughts?

Thanks


Thanks


.





.






.



--
Thanks
Zhang Chen

Re: [Qemu-devel] [PATCH 02/15] colo-compare: implement the process of checkpoint

2017-04-17 Thread Hailiang Zhang 

Hi Jason,

On 2017/4/14 14:38, Jason Wang wrote:


On 2017年04月14日 14:22, Hailiang Zhang wrote:

Hi Jason,

On 2017/4/14 13:57, Jason Wang wrote:

On 2017年02月22日 17:31, Zhang Chen wrote:

On 02/22/2017 11:42 AM, zhanghailiang wrote:

While do checkpoint, we need to flush all the unhandled packets,
By using the filter notifier mechanism, we can easily to notify
every compare object to do this process, which runs inside
of compare threads as a coroutine.

Hi~ Jason and Hailiang.

I will send a patch set later about colo-compare notify mechanism for
Xen like this patch.
I want to add a new chardev socket way in colo-comapre connect to Xen
colo, for notify
checkpoint or failover, Because We have no choice to use this way
communicate with Xen codes.
That's means we will have two notify mechanism.
What do you think about this?


Thanks
Zhang Chen

I was thinking the possibility of using similar way to for colo compare.
E.g can we use socket? This can saves duplicated codes more or less.

Since there are too many sockets used by filter and COLO, (Two unix
sockets and two
  tcp sockets for each vNIC), I don't want to introduce more ;) , but
i'm not sure if it is
possible to make it more flexible and optional, abstract these
duplicated codes,
pass the opened fd (No matter eventfd or socket fd ) as parameter, for
example.
Is this way acceptable ?

Thanks,
Hailiang

Yes, that's kind of what I want. We don't want to use two message
format. Passing a opened fd need management support, we still need a
fallback if there's no management on top. For qemu/kvm, we can do all
stuffs transparent to the cli by e.g socketpair() or others, but the key
is to have a unified message format.


After a deeper investigation, i think we can re-use most codes, since there is 
no
existing way to notify xen (no ?), we still needs notify chardev socket (Be 
used to notify xen, it is optional.)
(http://patchwork.ozlabs.org/patch/733431/ "COLO-compare: Add Xen notify chardev 
socket handler frame")

Besides, there is an existing qmp comand 'xen-colo-do-checkpoint', we can 
re-use it to notify
colo-compare objects and other filter objects to do checkpoint, for the 
opposite direction, we use
the notify chardev socket (Only for xen).

So the codes will be like:
diff --git a/migration/colo.c b/migration/colo.c
index 91da936..813c281 100644
--- a/migration/colo.c
+++ b/migration/colo.c
@@ -224,7 +224,19 @@ ReplicationStatus *qmp_query_xen_replication_status(Error 
**errp)

 void qmp_xen_colo_do_checkpoint(Error **errp)
 {
+Error *local_err = NULL;
+
 replication_do_checkpoint_all(errp);
+/* Notify colo-compare and other filters to do checkpoint */
+colo_notify_compares_event(NULL, COLO_CHECKPOINT, _err);
+if (local_err) {
+error_propagate(errp, local_err);
+return;
+}
+colo_notify_filters_event(COLO_CHECKPOINT, _err);
+if (local_err) {
+error_propagate(errp, local_err);
+}
 }

 static void colo_send_message(QEMUFile *f, COLOMessage msg,
diff --git a/net/colo-compare.c b/net/colo-compare.c
index 24e13f0..de975c5 100644
--- a/net/colo-compare.c
+++ b/net/colo-compare.c
@@ -391,6 +391,9 @@ static void colo_compare_inconsistent_notify(void)
 {
 notifier_list_notify(_compare_notifiers,
 migrate_get_current());
+if (s->notify_dev) {
+   /* Do something, notify remote side through notify dev */
+}
 }

 void colo_compare_register_notifier(Notifier *notify)

How about this scenario ？


Thoughts?

Thanks


Thanks


.





.

Re: [Qemu-devel] [PATCH 02/15] colo-compare: implement the process of checkpoint

2017-04-14 Thread Jason Wang 



On 2017年04月14日 14:22, Hailiang Zhang wrote:

Hi Jason,

On 2017/4/14 13:57, Jason Wang wrote:


On 2017年02月22日 17:31, Zhang Chen wrote:


On 02/22/2017 11:42 AM, zhanghailiang wrote:

While do checkpoint, we need to flush all the unhandled packets,
By using the filter notifier mechanism, we can easily to notify
every compare object to do this process, which runs inside
of compare threads as a coroutine.

Hi~ Jason and Hailiang.

I will send a patch set later about colo-compare notify mechanism for
Xen like this patch.
I want to add a new chardev socket way in colo-comapre connect to Xen
colo, for notify
checkpoint or failover, Because We have no choice to use this way
communicate with Xen codes.
That's means we will have two notify mechanism.
What do you think about this?


Thanks
Zhang Chen

I was thinking the possibility of using similar way to for colo compare.
E.g can we use socket? This can saves duplicated codes more or less.


Since there are too many sockets used by filter and COLO, (Two unix 
sockets and two
 tcp sockets for each vNIC), I don't want to introduce more ;) , but 
i'm not sure if it is
possible to make it more flexible and optional, abstract these 
duplicated codes,
pass the opened fd (No matter eventfd or socket fd ) as parameter, for 
example.

Is this way acceptable ?

Thanks,
Hailiang


Yes, that's kind of what I want. We don't want to use two message 
format. Passing a opened fd need management support, we still need a 
fallback if there's no management on top. For qemu/kvm, we can do all 
stuffs transparent to the cli by e.g socketpair() or others, but the key 
is to have a unified message format.


Thoughts?

Thanks




Thanks


.

Re: [Qemu-devel] [PATCH 02/15] colo-compare: implement the process of checkpoint

2017-04-14 Thread Hailiang Zhang 

Hi Jason,

On 2017/4/14 13:57, Jason Wang wrote:


On 2017年02月22日 17:31, Zhang Chen wrote:


On 02/22/2017 11:42 AM, zhanghailiang wrote:

While do checkpoint, we need to flush all the unhandled packets,
By using the filter notifier mechanism, we can easily to notify
every compare object to do this process, which runs inside
of compare threads as a coroutine.

Hi~ Jason and Hailiang.

I will send a patch set later about colo-compare notify mechanism for
Xen like this patch.
I want to add a new chardev socket way in colo-comapre connect to Xen
colo, for notify
checkpoint or failover, Because We have no choice to use this way
communicate with Xen codes.
That's means we will have two notify mechanism.
What do you think about this?


Thanks
Zhang Chen

I was thinking the possibility of using similar way to for colo compare.
E.g can we use socket? This can saves duplicated codes more or less.


Since there are too many sockets used by filter and COLO, (Two unix sockets and 
two
 tcp sockets for each vNIC), I don't want to introduce more ;) , but i'm not 
sure if it is
possible to make it more flexible and optional, abstract these duplicated codes,
pass the opened fd (No matter eventfd or socket fd ) as parameter, for example.
Is this way acceptable ?

Thanks,
Hailiang


Thanks


.

Re: [Qemu-devel] [PATCH 02/15] colo-compare: implement the process of checkpoint

2017-04-13 Thread Jason Wang 



On 2017年02月22日 17:31, Zhang Chen wrote:



On 02/22/2017 11:42 AM, zhanghailiang wrote:

While do checkpoint, we need to flush all the unhandled packets,
By using the filter notifier mechanism, we can easily to notify
every compare object to do this process, which runs inside
of compare threads as a coroutine.


Hi~ Jason and Hailiang.

I will send a patch set later about colo-compare notify mechanism for 
Xen like this patch.
I want to add a new chardev socket way in colo-comapre connect to Xen 
colo, for notify
checkpoint or failover, Because We have no choice to use this way 
communicate with Xen codes.

That's means we will have two notify mechanism.
What do you think about this?


Thanks
Zhang Chen 


I was thinking the possibility of using similar way to for colo compare. 
E.g can we use socket? This can saves duplicated codes more or less.


Thanks

Re: [Qemu-devel] [PATCH 02/15] colo-compare: implement the process of checkpoint

2017-02-22 Thread Zhang Chen 



On 02/23/2017 09:02 AM, Hailiang Zhang wrote:

Hi,

On 2017/2/22 17:31, Zhang Chen wrote:



On 02/22/2017 11:42 AM, zhanghailiang wrote:

While do checkpoint, we need to flush all the unhandled packets,
By using the filter notifier mechanism, we can easily to notify
every compare object to do this process, which runs inside
of compare threads as a coroutine.


Hi~ Jason and Hailiang.

I will send a patch set later about colo-compare notify mechanism for
Xen like this patch.
I want to add a new chardev socket way in colo-comapre connect to Xen
colo, for notify
checkpoint or failover, Because We have no choice to use this way
communicate with Xen codes.
That's means we will have two notify mechanism.
What do you think about this?



I don't think you need another mechanism, what you need to do is to
realize a qmp command which calls colo_notify_compares_event(),
It will not  return until the event (checkpoint or failover) be
handled by all compares. Will this satisfy your requirement ?


No, colo-frame notify colo-comapre can calls colo_notify_compares_event(),
That's OK, but colo-comapre notify colo-frame in Xen have some problem,
Xen's colo-frame needs a API that blocking and have a timeout to read 
colo-comapre's

notify, this timeout is the time of periodic checkpoint. In this patch set,
colo-compare just call colo_compare_inconsistent_notify() to 
non-blocking notify.
We can not realize a qmp command that Xen always polling that to get 
status of notify,

Qemu also can not accept to call qmp command for polling.


Thanks
Zhang Chen



Thanks,
Hailiang



Thanks
Zhang Chen



Cc: Jason Wang 
Signed-off-by: zhanghailiang 
Signed-off-by: Zhang Chen 
---
   net/colo-compare.c | 72 
++

   net/colo-compare.h | 20 +++
   2 files changed, 92 insertions(+)
   create mode 100644 net/colo-compare.h







.



--
Thanks
Zhang Chen

Re: [Qemu-devel] [PATCH 02/15] colo-compare: implement the process of checkpoint

2017-02-22 Thread Hailiang Zhang 

Hi,

On 2017/2/22 17:31, Zhang Chen wrote:



On 02/22/2017 11:42 AM, zhanghailiang wrote:

While do checkpoint, we need to flush all the unhandled packets,
By using the filter notifier mechanism, we can easily to notify
every compare object to do this process, which runs inside
of compare threads as a coroutine.


Hi~ Jason and Hailiang.

I will send a patch set later about colo-compare notify mechanism for
Xen like this patch.
I want to add a new chardev socket way in colo-comapre connect to Xen
colo, for notify
checkpoint or failover, Because We have no choice to use this way
communicate with Xen codes.
That's means we will have two notify mechanism.
What do you think about this?



I don't think you need another mechanism, what you need to do is to
realize a qmp command which calls colo_notify_compares_event(),
It will not  return until the event (checkpoint or failover) be
handled by all compares. Will this satisfy your requirement ?

Thanks,
Hailiang



Thanks
Zhang Chen



Cc: Jason Wang 
Signed-off-by: zhanghailiang 
Signed-off-by: Zhang Chen 
---
   net/colo-compare.c | 72 
++
   net/colo-compare.h | 20 +++
   2 files changed, 92 insertions(+)
   create mode 100644 net/colo-compare.h

Re: [Qemu-devel] [PATCH 02/15] colo-compare: implement the process of checkpoint

2017-02-22 Thread Zhang Chen 



On 02/22/2017 11:42 AM, zhanghailiang wrote:

While do checkpoint, we need to flush all the unhandled packets,
By using the filter notifier mechanism, we can easily to notify
every compare object to do this process, which runs inside
of compare threads as a coroutine.


Hi~ Jason and Hailiang.

I will send a patch set later about colo-compare notify mechanism for 
Xen like this patch.
I want to add a new chardev socket way in colo-comapre connect to Xen 
colo, for notify
checkpoint or failover, Because We have no choice to use this way 
communicate with Xen codes.

That's means we will have two notify mechanism.
What do you think about this?


Thanks
Zhang Chen



Cc: Jason Wang 
Signed-off-by: zhanghailiang 
Signed-off-by: Zhang Chen 
---
  net/colo-compare.c | 72 ++
  net/colo-compare.h | 20 +++
  2 files changed, 92 insertions(+)
  create mode 100644 net/colo-compare.h



--
Thanks
Zhang Chen

[Qemu-devel] [PATCH 02/15] colo-compare: implement the process of checkpoint

2017-02-21 Thread zhanghailiang 
While do checkpoint, we need to flush all the unhandled packets,
By using the filter notifier mechanism, we can easily to notify
every compare object to do this process, which runs inside
of compare threads as a coroutine.

Cc: Jason Wang 
Signed-off-by: zhanghailiang 
Signed-off-by: Zhang Chen 
---
 net/colo-compare.c | 72 ++
 net/colo-compare.h | 20 +++
 2 files changed, 92 insertions(+)
 create mode 100644 net/colo-compare.h

diff --git a/net/colo-compare.c b/net/colo-compare.c
index a6fc2ff..61a8ee4 100644
--- a/net/colo-compare.c
+++ b/net/colo-compare.c
@@ -29,17 +29,24 @@
 #include "qemu/sockets.h"
 #include "qapi-visit.h"
 #include "net/colo.h"
+#include "net/colo-compare.h"
 
 #define TYPE_COLO_COMPARE "colo-compare"
 #define COLO_COMPARE(obj) \
 OBJECT_CHECK(CompareState, (obj), TYPE_COLO_COMPARE)
 
+static QTAILQ_HEAD(, CompareState) net_compares =
+   QTAILQ_HEAD_INITIALIZER(net_compares);
+
 #define COMPARE_READ_LEN_MAX NET_BUFSIZE
 #define MAX_QUEUE_SIZE 1024
 
 /* TODO: Should be configurable */
 #define REGULAR_PACKET_CHECK_MS 3000
 
+static QemuMutex event_mtx = { .lock = PTHREAD_MUTEX_INITIALIZER };
+static QemuCond event_complete_cond = { .cond = PTHREAD_COND_INITIALIZER };
+static int event_unhandled_count;
 /*
   + CompareState ++
   |   |
@@ -86,6 +93,10 @@ typedef struct CompareState {
 
 GMainContext *worker_context;
 GMainLoop *compare_loop;
+/* Used for COLO to notify compare to do something */
+FilterNotifier *notifier;
+
+QTAILQ_ENTRY(CompareState) next;
 } CompareState;
 
 typedef struct CompareClass {
@@ -375,6 +386,11 @@ static void colo_compare_connection(void *opaque, void 
*user_data)
 while (!g_queue_is_empty(>primary_list) &&
!g_queue_is_empty(>secondary_list)) {
 pkt = g_queue_pop_tail(>primary_list);
+if (!pkt) {
+error_report("colo-compare pop pkt failed");
+return;
+}
+
 switch (conn->ip_proto) {
 case IPPROTO_TCP:
 result = g_queue_find_custom(>secondary_list,
@@ -496,6 +512,52 @@ static gboolean check_old_packet_regular(void *opaque)
 return TRUE;
 }
 
+/* Public API, Used for COLO frame to notify compare event */
+void colo_notify_compares_event(void *opaque, int event, Error **errp)
+{
+CompareState *s;
+int ret;
+
+qemu_mutex_lock(_mtx);
+QTAILQ_FOREACH(s, _compares, next) {
+ret = filter_notifier_set(s->notifier, event);
+if (ret < 0) {
+error_setg_errno(errp, -ret, "Failed to write value to eventfd");
+goto fail;
+}
+event_unhandled_count++;
+}
+/* Wait all compare thread to finish handling this event */
+while (event_unhandled_count) {
+qemu_cond_wait(_complete_cond, _mtx);
+}
+
+fail:
+qemu_mutex_unlock(_mtx);
+}
+
+static void colo_flush_packets(void *opaque, void *user_data);
+
+static void colo_compare_handle_event(void *opaque, int event)
+{
+FilterNotifier *notify = opaque;
+CompareState *s = notify->opaque;
+
+switch (event) {
+case COLO_CHECKPOINT:
+g_queue_foreach(>conn_list, colo_flush_packets, s);
+break;
+case COLO_FAILOVER:
+break;
+default:
+break;
+}
+qemu_mutex_lock(_mtx);
+event_unhandled_count--;
+qemu_cond_broadcast(_complete_cond);
+qemu_mutex_unlock(_mtx);
+}
+
 static void *colo_compare_thread(void *opaque)
 {
 CompareState *s = opaque;
@@ -516,8 +578,12 @@ static void *colo_compare_thread(void *opaque)
   (GSourceFunc)check_old_packet_regular, s, NULL);
 g_source_attach(timeout_source, s->worker_context);
 
+s->notifier = filter_noitifier_new(colo_compare_handle_event, s, NULL);
+g_source_attach(>notifier->source, s->worker_context);
+
 g_main_loop_run(s->compare_loop);
 
+g_source_unref(>notifier->source);
 g_source_unref(timeout_source);
 g_main_loop_unref(s->compare_loop);
 g_main_context_unref(s->worker_context);
@@ -660,6 +726,8 @@ static void colo_compare_complete(UserCreatable *uc, Error 
**errp)
 net_socket_rs_init(>pri_rs, compare_pri_rs_finalize);
 net_socket_rs_init(>sec_rs, compare_sec_rs_finalize);
 
+QTAILQ_INSERT_TAIL(_compares, s, next);
+
 g_queue_init(>conn_list);
 
 s->connection_track_table = g_hash_table_new_full(connection_key_hash,
@@ -726,6 +794,10 @@ static void colo_compare_finalize(Object *obj)
 g_main_loop_quit(s->compare_loop);
 qemu_thread_join(>thread);
 
+if (!QTAILQ_EMPTY(_compares)) {
+QTAILQ_REMOVE(_compares, s, next);
+}
+
 /* Release all unhandled packets after compare thead exited */
 g_queue_foreach(>conn_list, colo_flush_packets, s);
 
diff --git a/net/colo-compare.h b/net/colo-compare.h
new file mode 100644
index 000..ed823ed
---