Re: Proposal for a regular upstream performance testing

2022-03-28 Thread Lukáš Doktor
Dne 28. 03. 22 v 11:57 Stefan Hajnoczi napsal(a): > On Mon, Mar 28, 2022 at 08:18:43AM +0200, Lukáš Doktor wrote: >> Hello Stefan, folks, >> >> I seem to have another hit, an improvement actually and it seems to be >> bisected all the way to you, Stefan. Let me use this as another example of >>

Re: Proposal for a regular upstream performance testing

2022-03-28 Thread Stefan Hajnoczi
On Mon, Mar 28, 2022 at 08:18:43AM +0200, Lukáš Doktor wrote: > Hello Stefan, folks, > > I seem to have another hit, an improvement actually and it seems to be > bisected all the way to you, Stefan. Let me use this as another example of > how such process could look like and we can use this to

Re: Proposal for a regular upstream performance testing

2022-03-28 Thread Lukáš Doktor
Hello Stefan, folks, I seem to have another hit, an improvement actually and it seems to be bisected all the way to you, Stefan. Let me use this as another example of how such process could look like and we can use this to hammer-out the details like via what means to submit the request, whom

Re: Proposal for a regular upstream performance testing

2022-03-22 Thread Stefan Hajnoczi
On Mon, Mar 21, 2022 at 11:29:42AM +0100, Lukáš Doktor wrote: > Hello Stefan, > > Dne 21. 03. 22 v 10:42 Stefan Hajnoczi napsal(a): > > On Mon, Mar 21, 2022 at 09:46:12AM +0100, Lukáš Doktor wrote: > >> Dear qemu developers, > >> > >> you might remember the "replied to" email from a bit over year

Re: Proposal for a regular upstream performance testing

2022-03-21 Thread Lukáš Doktor
Hello Stefan, Dne 21. 03. 22 v 10:42 Stefan Hajnoczi napsal(a): > On Mon, Mar 21, 2022 at 09:46:12AM +0100, Lukáš Doktor wrote: >> Dear qemu developers, >> >> you might remember the "replied to" email from a bit over year ago to raise >> a discussion about a qemu performance regression CI. On

Re: Proposal for a regular upstream performance testing

2022-03-21 Thread Stefan Hajnoczi
On Mon, Mar 21, 2022 at 09:46:12AM +0100, Lukáš Doktor wrote: > Dear qemu developers, > > you might remember the "replied to" email from a bit over year ago to raise a > discussion about a qemu performance regression CI. On KVM forum I presented >

Re: Proposal for a regular upstream performance testing

2022-03-21 Thread Lukáš Doktor
Dear qemu developers, you might remember the "replied to" email from a bit over year ago to raise a discussion about a qemu performance regression CI. On KVM forum I presented https://www.youtube.com/watch?v=Cbm3o4ACE3Y=PLbzoR-pLrL6q4ZzA4VRpy42Ua4-D2xHUR=9 some details about my testing

RE: Proposal for a regular upstream performance testing

2020-12-02 Thread Chenqun (kuhn)
> On Tue, Dec 01, 2020 at 01:06:35PM +0100, Lukáš Doktor wrote: > > Dne 01. 12. 20 v 11:22 Stefan Hajnoczi napsal(a): > > > On Tue, Dec 01, 2020 at 09:05:49AM +0100, Lukáš Doktor wrote: > > > > Dne 30. 11. 20 v 14:25 Stefan Hajnoczi napsal(a): > > > > > On Thu, Nov 26, 2020 at 09:10:14AM +0100,

RE: Proposal for a regular upstream performance testing

2020-12-02 Thread Chenqun (kuhn)
> -Original Message- > From: Qemu-devel > [mailto:qemu-devel-bounces+kuhn.chenqun=huawei@nongnu.org] On > Behalf Of Luká? Doktor > Sent: Thursday, November 26, 2020 4:10 PM > To: QEMU Developers > Cc: Charles Shih ; Aleksandar Markovic > ; Stefan Hajnoczi > > Subject: Proposal for a

Re: Proposal for a regular upstream performance testing

2020-12-01 Thread Stefan Hajnoczi
On Tue, Dec 01, 2020 at 01:06:35PM +0100, Lukáš Doktor wrote: > Dne 01. 12. 20 v 11:22 Stefan Hajnoczi napsal(a): > > On Tue, Dec 01, 2020 at 09:05:49AM +0100, Lukáš Doktor wrote: > > > Dne 30. 11. 20 v 14:25 Stefan Hajnoczi napsal(a): > > > > On Thu, Nov 26, 2020 at 09:10:14AM +0100, Lukáš Doktor

Re: Proposal for a regular upstream performance testing

2020-12-01 Thread Lukáš Doktor
Dne 01. 12. 20 v 11:22 Stefan Hajnoczi napsal(a): On Tue, Dec 01, 2020 at 09:05:49AM +0100, Lukáš Doktor wrote: Dne 30. 11. 20 v 14:25 Stefan Hajnoczi napsal(a): On Thu, Nov 26, 2020 at 09:10:14AM +0100, Lukáš Doktor wrote: What is the minimal environment needed for bare metal hosts? Not

Re: Proposal for a regular upstream performance testing

2020-12-01 Thread Stefan Hajnoczi
On Tue, Dec 01, 2020 at 09:05:49AM +0100, Lukáš Doktor wrote: > Dne 30. 11. 20 v 14:25 Stefan Hajnoczi napsal(a): > > On Thu, Nov 26, 2020 at 09:10:14AM +0100, Lukáš Doktor wrote: > > What is the minimal environment needed for bare metal hosts? > > > > Not sure what you mean by that. For

Re: Proposal for a regular upstream performance testing

2020-12-01 Thread Lukáš Doktor
Dne 30. 11. 20 v 14:25 Stefan Hajnoczi napsal(a): On Thu, Nov 26, 2020 at 09:10:14AM +0100, Lukáš Doktor wrote: The problem with those is that we can not simply use travis/gitlab/... machines for running those tests, because we are measuring in-guest actual performance. We can't just stop the

Re: Proposal for a regular upstream performance testing

2020-11-30 Thread Lukáš Doktor
Dne 30. 11. 20 v 14:23 Stefan Hajnoczi napsal(a): On Thu, Nov 26, 2020 at 09:43:38AM +, Daniel P. Berrangé wrote: On Thu, Nov 26, 2020 at 09:10:14AM +0100, Lukáš Doktor wrote: Ideally the community should have a way to also issue their custom builds in order to verify their patches so they

Re: Proposal for a regular upstream performance testing

2020-11-30 Thread Stefan Hajnoczi
On Thu, Nov 26, 2020 at 09:43:38AM +, Daniel P. Berrangé wrote: > On Thu, Nov 26, 2020 at 09:10:14AM +0100, Lukáš Doktor wrote: > > Ideally the community should have a way to also issue their custom builds > > in order to verify their patches so they can debug and address issues > > better

Re: Proposal for a regular upstream performance testing

2020-11-30 Thread Stefan Hajnoczi
On Thu, Nov 26, 2020 at 09:10:14AM +0100, Lukáš Doktor wrote: > The problem with those is that we can not simply use travis/gitlab/... > machines for running those tests, because we are measuring in-guest actual > performance. We can't just stop the time when the machine decides to schedule >

Re: Proposal for a regular upstream performance testing

2020-11-26 Thread Lukáš Doktor
Dne 26. 11. 20 v 10:43 Daniel P. Berrangé napsal(a): On Thu, Nov 26, 2020 at 09:10:14AM +0100, Lukáš Doktor wrote: How === This is a tough question. Ideally this should be a standalone service that would only notify the author of the patch that caused the change with a bunch of useful data so

Re: Proposal for a regular upstream performance testing

2020-11-26 Thread Lukáš Doktor
Dne 26. 11. 20 v 11:17 Peter Maydell napsal(a): On Thu, 26 Nov 2020 at 08:13, Lukáš Doktor wrote: The goal of this initiative is to detect system-wide performance regressions as well as improvements early, ideally pin-point the individual commits and notify people that they should fix things.

Re: Proposal for a regular upstream performance testing

2020-11-26 Thread Peter Maydell
On Thu, 26 Nov 2020 at 08:13, Lukáš Doktor wrote: > The goal of this initiative is to detect system-wide performance > regressions as well as improvements early, ideally pin-point the > individual commits and notify people that they should fix things. > All in upstream and ideally with least

Re: Proposal for a regular upstream performance testing

2020-11-26 Thread Daniel P . Berrangé
On Thu, Nov 26, 2020 at 09:10:14AM +0100, Lukáš Doktor wrote: > How > === > > This is a tough question. Ideally this should be a standalone service that > would only notify the author of the patch that caused the change with a > bunch of useful data so they can either address the issue or just be

Re: Proposal for a regular upstream performance testing

2020-11-26 Thread Jason Wang
On 2020/11/26 下午4:10, Lukáš Doktor wrote: Hello guys, I had been around qemu on the Avocado-vt side for quite some time and a while ago I shifted my focus on performance testing. Currently I am not aware of any upstream CI that would continuously monitor the upstream qemu performance and