Re: [Piglit] [PATCH] drawoverhead: new microbenchmark

2017-05-22 Thread Brian Paul 

LGTM.
Reviewed-by: Brian Paul 

On 04/30/2017 05:58 PM, Marek Olšák wrote:

From: Marek Olšák 

Based on a benchmark from mesa/demos, but rewritten and extended.
It's a benchmark expected to be run separately, not a piglit test.
So why piglit? Because it's a good framework for writing apps like this.

mesa_glthread won't show an improvement here, because there is no app
overhead.

This is what the output looks like. The percentage is relative to
the first test of the given draw call.

The obvious thing there is that enabled vertex attribs decrease
Mesa performance even if there are no state changes.

Using Core profile.
Draw calls per second:
DrawElements ( 1 VBOs, 0 UBOs,  0 Tex) w/ no state change:  5.71 
million (100.0%)
DrawElements ( 4 VBOs, 0 UBOs,  0 Tex) w/ no state change:  5.18 
million (90.8%)
DrawElements (16 VBOs, 0 UBOs,  0 Tex) w/ no state change:  3.65 
million (63.9%)
DrawElements ( 1 VBOs, 0 UBOs, 16 Tex) w/ no state change:  5.71 
million (100.0%)
DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ no state change:  5.78 
million (101.2%)
DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ shader program change:220.11 
thousand (3.9%)
DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ vertex attrib change: 1.06 
million (18.5%)
DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ 1 texture change: 483.27 
thousand (8.5%)
DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ 8 textures change:291.20 
thousand (5.1%)
DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ 1 UBO change: 1.84 
million (32.3%)
DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ 4 UBOs change:1.12 
million (19.7%)
DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ few uniforms / 1 change:  2.27 
million (39.8%)
DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ many uniforms / 1 change: 966.00 
thousand (16.9%)
DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ blend enable change:  1.37 
million (24.0%)
DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ depth enable change:  1.86 
million (32.6%)
DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ stencil enable change:1.66 
million (29.0%)
DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ scissor enable change:1.09 
million (19.1%)
DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ MSAA enable change:   1.94 
million (34.0%)
DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ cull face enable change:  1.56 
million (27.3%)
DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ FB sRGB enable change:200.81 
thousand (3.5%)
DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ shader program change:186.92 
thousand (3.3%)
DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ vertex attrib change: 638.49 
thousand (11.2%)
DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ 1 texture change: 452.39 
thousand (7.9%)
DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ 8 textures change:278.79 
thousand (4.9%)
DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ 1 UBO change: 1.47 
million (25.7%)
DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ 4 UBOs change:974.30 
thousand (17.1%)
DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ few uniforms / 1 change:  1.79 
million (31.3%)
DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ many uniforms / 1 change: 853.07 
thousand (14.9%)
DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ blend enable change:  1.16 
million (20.3%)
DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ depth enable change:  1.49 
million (26.2%)
DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ stencil enable change:1.35 
million (23.7%)
DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ scissor enable change:946.45 
thousand (16.6%)
DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ MSAA enable change:   1.62 
million (28.3%)
DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ cull face enable change:  1.29 
million (22.5%)
DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ FB sRGB enable change:126.44 
thousand (2.2%)
DrawArrays ( 1 VBOs, 0 UBOs,  0 Tex) w/ no state change:  8.02 
million (100.0%)
DrawArrays ( 4 VBOs, 0 UBOs,  0 Tex) w/ no state change:  7.14 
million (89.0%)
DrawArrays (16 VBOs, 0 UBOs,  0 Tex) w/ no state change:  4.26 
million (53.0%)
DrawArrays ( 1 VBOs, 0 UBOs, 16 Tex) w/ no state change:  7.89 
million (98.4%)
DrawArrays ( 1 VBOs, 4 UBOs,  8 Tex) w/ no state change:  8.01 
million (99.9%)
DrawArrays ( 1 VBOs, 4 UBOs,  8 Tex) w/ shader program change:221.09 
thousand (2.8%)
DrawArrays ( 1 VBOs, 4 UBOs,  8 Tex) w/ vertex attrib change: 1.13 
million (14.1%)
DrawArrays ( 1 VBOs, 4 UBOs,  8 Tex) w/ 1 texture change: 500.25 
thousand (6.2%)
DrawArrays ( 1 VBOs, 4 UBOs,  8 Tex) w/ 8 textures change:294.30 
thousand (3.7%)
DrawArrays ( 1 VBOs, 4 UBOs,  8 Tex) w/ 1 UBO change: 2.02 
million (25.2%)
DrawArrays ( 1 VBOs, 4 UBOs,  8 Tex) w/ 4 UBOs

Re: [Piglit] [PATCH] drawoverhead: new microbenchmark

2017-05-22 Thread Brian Paul 

On 05/15/2017 08:05 AM, Marek Olšák wrote:

On Thu, May 11, 2017 at 9:49 PM, Eric Anholt  wrote:

Marek Olšák  writes:


From: Marek Olšák 

Based on a benchmark from mesa/demos, but rewritten and extended.
It's a benchmark expected to be run separately, not a piglit test.
So why piglit? Because it's a good framework for writing apps like this.

mesa_glthread won't show an improvement here, because there is no app
overhead.

This is what the output looks like. The percentage is relative to
the first test of the given draw call.

The obvious thing there is that enabled vertex attribs decrease
Mesa performance even if there are no state changes.


Since nobody else has replied,

I think piglit is the wrong place for this.  I agree that it's sorta
convenient, but mesa-demos or glmark2 are the right place.


It can be moved to mesa/demos or glmark2 if/when they get support for
piglit's dispatch and multiple window systems including gbm/drm, so
that the code size of the test remains the same.


IMHO, I think it's fine to add performance tests to Piglit (I proposed a 
few years ago).  As Marek said, the infrastructure is there and it would 
be wasted effort to duplicate it elsewhere.


-Brian

___
Piglit mailing list
Piglit@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/piglit

Re: [Piglit] [PATCH] drawoverhead: new microbenchmark

2017-05-15 Thread Marek Olšák 
On Thu, May 11, 2017 at 9:49 PM, Eric Anholt  wrote:
> Marek Olšák  writes:
>
>> From: Marek Olšák 
>>
>> Based on a benchmark from mesa/demos, but rewritten and extended.
>> It's a benchmark expected to be run separately, not a piglit test.
>> So why piglit? Because it's a good framework for writing apps like this.
>>
>> mesa_glthread won't show an improvement here, because there is no app
>> overhead.
>>
>> This is what the output looks like. The percentage is relative to
>> the first test of the given draw call.
>>
>> The obvious thing there is that enabled vertex attribs decrease
>> Mesa performance even if there are no state changes.
>
> Since nobody else has replied,
>
> I think piglit is the wrong place for this.  I agree that it's sorta
> convenient, but mesa-demos or glmark2 are the right place.

It can be moved to mesa/demos or glmark2 if/when they get support for
piglit's dispatch and multiple window systems including gbm/drm, so
that the code size of the test remains the same.

Marek
___
Piglit mailing list
Piglit@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/piglit

Re: [Piglit] [PATCH] drawoverhead: new microbenchmark

2017-05-14 Thread Timothy Arceri 



On 12/05/17 05:49, Eric Anholt wrote:

Marek Olšák  writes:


From: Marek Olšák 

Based on a benchmark from mesa/demos, but rewritten and extended.
It's a benchmark expected to be run separately, not a piglit test.
So why piglit? Because it's a good framework for writing apps like this.

mesa_glthread won't show an improvement here, because there is no app
overhead.

This is what the output looks like. The percentage is relative to
the first test of the given draw call.

The obvious thing there is that enabled vertex attribs decrease
Mesa performance even if there are no state changes.


Since nobody else has replied,

I think piglit is the wrong place for this.  I agree that it's sorta
convenient, but mesa-demos or glmark2 are the right place.


I'm not totally against this. It would be nice to leverage more of the 
piglit framework/tools to be able to run a group of perf tests and 
generate some kind of comparison automatically. That way way we could 
have an easy tools for comparing releases etc. I know Intel has an 
internal setup for this type of thing, but having something in the test 
suite all mesa devs use would be handy.


Anyway I've given this benchmark a run with my KHR_no_error changes and 
I'm seeing a nice jump in a bunch of the tests.


The biggest jump is:

DrawArrays(16 VBOs, 0 UBOs,  0 Tex) w/ no state change: 5.41 million (49.3%)

DrawArrays(16 VBOs, 0 UBOs,  0 Tex) w/ no state change: 16.33 million 
(109.1%)


But there is a general increase across the majority of tests which is 
encouraging to see as I still haven't been able to measure much change 
in the games I've been trying. I suspect I'll need to have coverage 
across most of the api a game uses before I see much change.

___
Piglit mailing list
Piglit@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/piglit

Re: [Piglit] [PATCH] drawoverhead: new microbenchmark

2017-05-11 Thread Eric Anholt 
Marek Olšák  writes:

> From: Marek Olšák 
>
> Based on a benchmark from mesa/demos, but rewritten and extended.
> It's a benchmark expected to be run separately, not a piglit test.
> So why piglit? Because it's a good framework for writing apps like this.
>
> mesa_glthread won't show an improvement here, because there is no app
> overhead.
>
> This is what the output looks like. The percentage is relative to
> the first test of the given draw call.
>
> The obvious thing there is that enabled vertex attribs decrease
> Mesa performance even if there are no state changes.

Since nobody else has replied,

I think piglit is the wrong place for this.  I agree that it's sorta
convenient, but mesa-demos or glmark2 are the right place.


signature.asc
Description: PGP signature
___
Piglit mailing list
Piglit@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/piglit

[Piglit] [PATCH] drawoverhead: new microbenchmark

2017-04-30 Thread Marek Olšák 
From: Marek Olšák 

Based on a benchmark from mesa/demos, but rewritten and extended.
It's a benchmark expected to be run separately, not a piglit test.
So why piglit? Because it's a good framework for writing apps like this.

mesa_glthread won't show an improvement here, because there is no app
overhead.

This is what the output looks like. The percentage is relative to
the first test of the given draw call.

The obvious thing there is that enabled vertex attribs decrease
Mesa performance even if there are no state changes.

Using Core profile.
Draw calls per second:
   DrawElements ( 1 VBOs, 0 UBOs,  0 Tex) w/ no state change:  5.71 
million (100.0%)
   DrawElements ( 4 VBOs, 0 UBOs,  0 Tex) w/ no state change:  5.18 
million (90.8%)
   DrawElements (16 VBOs, 0 UBOs,  0 Tex) w/ no state change:  3.65 
million (63.9%)
   DrawElements ( 1 VBOs, 0 UBOs, 16 Tex) w/ no state change:  5.71 
million (100.0%)
   DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ no state change:  5.78 
million (101.2%)
   DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ shader program change:220.11 
thousand (3.9%)
   DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ vertex attrib change: 1.06 
million (18.5%)
   DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ 1 texture change: 483.27 
thousand (8.5%)
   DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ 8 textures change:291.20 
thousand (5.1%)
   DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ 1 UBO change: 1.84 
million (32.3%)
   DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ 4 UBOs change:1.12 
million (19.7%)
   DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ few uniforms / 1 change:  2.27 
million (39.8%)
   DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ many uniforms / 1 change: 966.00 
thousand (16.9%)
   DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ blend enable change:  1.37 
million (24.0%)
   DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ depth enable change:  1.86 
million (32.6%)
   DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ stencil enable change:1.66 
million (29.0%)
   DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ scissor enable change:1.09 
million (19.1%)
   DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ MSAA enable change:   1.94 
million (34.0%)
   DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ cull face enable change:  1.56 
million (27.3%)
   DrawElements ( 1 VBOs, 4 UBOs,  8 Tex) w/ FB sRGB enable change:200.81 
thousand (3.5%)
   DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ shader program change:186.92 
thousand (3.3%)
   DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ vertex attrib change: 638.49 
thousand (11.2%)
   DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ 1 texture change: 452.39 
thousand (7.9%)
   DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ 8 textures change:278.79 
thousand (4.9%)
   DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ 1 UBO change: 1.47 
million (25.7%)
   DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ 4 UBOs change:974.30 
thousand (17.1%)
   DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ few uniforms / 1 change:  1.79 
million (31.3%)
   DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ many uniforms / 1 change: 853.07 
thousand (14.9%)
   DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ blend enable change:  1.16 
million (20.3%)
   DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ depth enable change:  1.49 
million (26.2%)
   DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ stencil enable change:1.35 
million (23.7%)
   DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ scissor enable change:946.45 
thousand (16.6%)
   DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ MSAA enable change:   1.62 
million (28.3%)
   DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ cull face enable change:  1.29 
million (22.5%)
   DrawElements (16 VBOs, 4 UBOs,  8 Tex) w/ FB sRGB enable change:126.44 
thousand (2.2%)
   DrawArrays ( 1 VBOs, 0 UBOs,  0 Tex) w/ no state change:  8.02 
million (100.0%)
   DrawArrays ( 4 VBOs, 0 UBOs,  0 Tex) w/ no state change:  7.14 
million (89.0%)
   DrawArrays (16 VBOs, 0 UBOs,  0 Tex) w/ no state change:  4.26 
million (53.0%)
   DrawArrays ( 1 VBOs, 0 UBOs, 16 Tex) w/ no state change:  7.89 
million (98.4%)
   DrawArrays ( 1 VBOs, 4 UBOs,  8 Tex) w/ no state change:  8.01 
million (99.9%)
   DrawArrays ( 1 VBOs, 4 UBOs,  8 Tex) w/ shader program change:221.09 
thousand (2.8%)
   DrawArrays ( 1 VBOs, 4 UBOs,  8 Tex) w/ vertex attrib change: 1.13 
million (14.1%)
   DrawArrays ( 1 VBOs, 4 UBOs,  8 Tex) w/ 1 texture change: 500.25 
thousand (6.2%)
   DrawArrays ( 1 VBOs, 4 UBOs,  8 Tex) w/ 8 textures change:294.30 
thousand (3.7%)
   DrawArrays ( 1 VBOs, 4 UBOs,  8 Tex) w/ 1 UBO change: 2.02 
million (25.2%)
   DrawArrays ( 1 VBOs, 4 UBOs,  8 Tex) w/ 4 UBOs change:1.18 
million (14.7%)
   DrawArrays ( 1 VBOs, 4 UBOs,  8 Tex) w/ few uniforms / 1 change:  2.28 
million (28.4%)
   DrawArrays