Brian Aker wrote:
Hi!
Once upon a time there was a version of sql-bench that would save it's
results to a database, and there was an app to diff it. I wonder where
that might have went...
Getting a simple message of "this was slower from the last run" would be
useful. So just print out the differences. One other thought... give out
the schema and see who could write a php/etc that would just graph the
results.
No need. The infrastructure for this is already in drizzle-automation,
so we should use that...
I can work on this next week, or possibly over the weekend. Shouldn't
be difficult. We just need to create the schema tables for what we want
to store, and it should take me a couple hours to write the regexes to
strip the data and insert into the schema.
Cheers,
Jay
Cheers,
-Brian
On Aug 11, 2009, at 4:03 PM, Lee Bieber wrote:
Thanks very much to Patrick for getting sql-bench setup up and running
again, we have a basic run now automated through Hudson. Attached
(build-1113.txt) is typical output from a sql-bench run. I'd like to
get input on what we want to see in an email after each run. Diff'ing
the results with a previous run presents too much info since the times
are so small. A few ideas:
* Start with a compare of just theTOTALS line with previous runs
* Take the last section "Totals per operation" to the end and sort
it with a number previous runs so you would see something like the
second attachment (which only has data for 2 runs)
* Create a master file to compare against, if any times go over by a
certain percentage, raise the red flag
Other suggestions/ideas??
-Lee
Benchmark DBD suite: 2.15
Date of test: 2009-08-11 12:22:57
Running tests on: Linux 2.6.28-11-server x86_64
Arguments: --connect-options=port=9306
Comments:
Limits from:
Server version: Drizzle 2009.08.1113
Optimization: None
Hardware:
alter-table: Total time: 29 wallclock secs ( 0.03 usr 0.03 sys +
0.00 cusr 0.00 csys = 0.06 CPU)
ATIS: Total time: 16 wallclock secs ( 4.74 usr 0.28 sys + 0.00 cusr
0.00 csys = 5.02 CPU)
big-tables: Total time: 20 wallclock secs ( 9.14 usr 0.20 sys + 0.00
cusr 0.00 csys = 9.34 CPU)
connect: Total time: 19 wallclock secs ( 6.26 usr 3.80 sys + 0.00
cusr 0.00 csys = 10.06 CPU)
create: Total time: 82 wallclock secs ( 5.18 usr 2.16 sys + 0.00
cusr 0.00 csys = 7.34 CPU)
insert: Total time: 3734 wallclock secs (270.70 usr 45.50 sys + 0.00
cusr 0.00 csys = 316.20 CPU)
select: Total time: 319 wallclock secs (28.24 usr 3.05 sys + 0.00
cusr 0.00 csys = 31.29 CPU)
transactions: Total time: 16 wallclock secs ( 2.89 usr 1.40 sys +
0.00 cusr 0.00 csys = 4.29 CPU)
wisconsin: Total time: 12 wallclock secs ( 2.46 usr 0.68 sys + 0.00
cusr 0.00 csys = 3.14 CPU)
All 9 test executed successfully
Totals per operation:
Operation seconds usr sys cpu tests
alter_table_add 13.00 0.01 0.00
0.01 100
alter_table_drop 12.00 0.00 0.00
0.00 91
connect 2.00 0.84 0.36 1.20
2000
connect+select_1_row 2.00 1.01 0.56 1.57
2000
connect+select_simple 2.00 1.22 0.72 1.94
2000
count 16.00 0.01 0.00
0.01 100
count_distinct 5.00 0.11 0.00 0.11
1000
count_distinct_2 12.00 0.21 0.02 0.23
1000
count_distinct_big 20.00 4.24 0.00
4.24 120
count_distinct_group 13.00 0.72 0.05 0.77
1000
count_distinct_group_on_key 10.00 0.21 0.01 0.22
1000
count_distinct_group_on_key_parts 12.00 0.77 0.05 0.82
1000
count_distinct_key_prefix 4.00 0.15 0.02 0.17
1000
count_group_on_key_parts 8.00 0.35 0.04 0.39
1000
count_on_key 72.00 5.50 0.95 6.45
50100
create+drop 19.00 1.43 0.58 2.01
10000
create_MANY_tables 17.00 1.26 0.44 1.70
10000
create_index 2.00 0.00 0.00
0.00 8
create_key+drop 22.00 1.70 0.74 2.44
10000
create_table 0.00 0.00 0.00
0.00 31
delete_all_many_keys 1065.00 0.01 0.00
0.01 1
delete_autocommit 3.00 0.19 0.14 0.33
10000
delete_big 0.00 0.00 0.00
0.00 1
delete_big_many_keys 1065.00 0.01 0.00
0.01 128
delete_commit 1.00 0.33 0.13 0.46
10000
delete_key 2.00 0.29 0.14 0.43
10000
delete_range 8.00 0.00 0.00
0.00 12
delete_rollback 2.00 0.24 0.14
0.38 100
drop_index 2.00 0.00 0.00
0.00 8
drop_table 0.00 0.00 0.00
0.00 28
drop_table_when_MANY_tables 17.00 0.49 0.20 0.69
10000
insert 92.00 8.19 6.69 14.88
350768
insert_autocommit 3.00 0.45 0.22 0.67
10000
insert_commit 1.00 0.29 0.21 0.50
10000
insert_duplicates 14.00 2.91 1.67 4.58
100000
insert_key 523.00 5.69 2.11 7.80
100000
insert_many_fields 4.00 0.17 0.02 0.19
2000
insert_rollback 1.00 0.37 0.11
0.48 100
insert_select_1_key 4.00 0.00 0.00
0.00 1
insert_select_2_keys 6.00 0.00 0.00
0.00 1
min_max 9.00 0.01 0.00
0.01 60
min_max_on_key 18.00 10.03 1.30 11.33
85000
multiple_value_insert 2.00 0.12 0.01 0.13
100000
once_prepared_select 22.00 5.56 3.53 9.09
100000
order_by_big 15.00 12.17 0.04
12.21 10
order_by_big_key 13.00 11.54 0.22
11.76 10
order_by_big_key2 11.00 10.25 0.05
10.30 10
order_by_big_key_desc 13.00 11.68 0.20
11.88 10
order_by_big_key_diff 13.00 11.41 0.10
11.51 10
order_by_big_key_prefix 13.00 10.28 0.03
10.31 10
order_by_key2_diff 1.00 0.83 0.01
0.84 500
order_by_key_prefix 1.00 0.45 0.03
0.48 500
order_by_range 1.00 0.50 0.02
0.52 500
outer_join 23.00 0.00 0.00
0.00 10
outer_join_found 21.00 0.00 0.00
0.00 10
outer_join_not_found 21.00 0.02 0.00
0.02 500
outer_join_on_key 13.00 0.00 0.00
0.00 10
prepared_select 26.00 10.86 3.20 14.06
100000
select_1_row 2.00 0.68 0.33 1.01
20000
select_1_row_cache 2.00 0.47 0.36 0.83
20000
select_2_rows 3.00 0.54 0.30 0.84
20000
select_big 12.00 10.49 0.15
10.64 80
select_big_str 1.00 0.46 0.13 0.59
2000
select_cache 55.00 1.03 0.18 1.21
10000
select_cache2 61.00 1.21 0.20 1.41
10000
select_column+column 2.00 0.35 0.33 0.68
20000
select_diff_key 0.00 0.05 0.01
0.06 500
select_distinct 3.00 0.96 0.00
0.96 800
select_group 18.00 0.65 0.07 0.72
2911
select_group_when_MANY_tables 7.00 0.30 0.20 0.50
10000
select_join 1.00 0.25 0.01
0.26 100
select_key 44.00 22.09 3.11 25.20
200000
select_key2 49.00 22.73 3.19 25.92
200000
select_key2_return_key 44.00 19.47 3.27 22.74
200000
select_key2_return_prim 46.00 19.27 3.19 22.46
200000
select_key_prefix 53.00 29.17 3.25 32.42
200000
select_key_prefix_join 5.00 2.33 0.00
2.33 100
select_key_return_key 40.00 19.96 2.82 22.78
200000
select_many_fields 16.00 8.97 0.18 9.15
2000
select_range 49.00 4.59 0.23
4.82 410
select_range_key2 7.00 2.61 0.34 2.95
25010
select_range_prefix 7.00 2.14 0.28 2.42
25010
select_simple 1.00 0.29 0.35 0.64
20000
select_simple_cache 2.00 0.40 0.36 0.76
20000
select_simple_join 0.00 0.34 0.02
0.36 500
update_autocommit 3.00 0.36 0.15 0.51
10000
update_big 45.00 0.01 0.00
0.01 10
update_commit 1.00 0.33 0.13 0.46
10000
update_of_key 31.00 1.67 0.86 2.53
50000
update_of_key_big 44.00 0.00 0.03
0.03 501
update_of_primary_key_many_keys 1196.00 0.03 0.01
0.04 256
update_rollback 1.00 0.33 0.17
0.50 100
update_with_key 93.00 7.86 5.44 13.30
300000
update_with_key_prefix 36.00 4.86 2.18 7.04
100000
wisc_benchmark 3.00 1.72 0.11
1.83 114
TOTALS 5295.00 324.05 57.00 381.05
2974250
Operation seconds usr sys cpu
tests
alter_table_add 13.00 0.01 0.00
0.01 100
alter_table_add 15.00 0.00 0.01
0.01 100
alter_table_drop 12.00 0.00 0.00
0.00 91
alter_table_drop 13.00 0.02 0.01
0.03 91
connect 2.00 0.59 0.33 0.92
2000
connect 2.00 0.84 0.36 1.20
2000
connect+select_1_row 2.00 1.01 0.56 1.57
2000
connect+select_1_row 2.00 1.04 0.55 1.59
2000
connect+select_simple 2.00 1.17 0.66 1.83
2000
connect+select_simple 2.00 1.22 0.72 1.94
2000
count 16.00 0.01 0.00
0.01 100
count 16.00 0.04 0.00
0.04 100
count_distinct 5.00 0.11 0.00 0.11
1000
count_distinct 7.00 0.05 0.02 0.07
1000
count_distinct_2 12.00 0.21 0.02 0.23
1000
count_distinct_2 14.00 0.34 0.06 0.40
1000
count_distinct_big 20.00 4.24 0.00
4.24 120
count_distinct_big 21.00 4.22 0.02
4.24 120
count_distinct_group 11.00 0.28 0.04 0.32
1000
count_distinct_group 13.00 0.72 0.05 0.77
1000
count_distinct_group_on_key 9.00 0.16 0.06 0.22
1000
count_distinct_group_on_key 10.00 0.21 0.01 0.22
1000
count_distinct_group_on_key_parts 10.00 0.00 0.00 0.00
1000
count_distinct_group_on_key_parts 12.00 0.77 0.05 0.82
1000
count_distinct_key_prefix 4.00 0.11 0.03 0.14
1000
count_distinct_key_prefix 4.00 0.15 0.02 0.17
1000
count_group_on_key_parts 8.00 0.35 0.04 0.39
1000
count_group_on_key_parts 8.00 0.49 0.02 0.51
1000
count_on_key 72.00 5.50 0.95 6.45
50100
count_on_key 73.00 5.95 0.90 6.85
50100
create+drop 19.00 1.39 0.64 2.03
10000
create+drop 19.00 1.43 0.58 2.01
10000
create_MANY_tables 15.00 1.15 0.42 1.57
10000
create_MANY_tables 17.00 1.26 0.44 1.70
10000
create_index 2.00 0.00 0.00
0.00 8
create_index 2.00 0.00 0.00
0.00 8
create_key+drop 22.00 1.65 0.56 2.21
10000
create_key+drop 22.00 1.70 0.74 2.44
10000
create_table 0.00 0.00 0.00
0.00 31
create_table 0.00 0.01 0.01
0.02 31
delete_all_many_keys 1065.00 0.01 0.00
0.01 1
delete_all_many_keys 1117.00 0.01 0.00
0.01 1
delete_autocommit 3.00 0.19 0.14 0.33
10000
delete_autocommit 3.00 0.22 0.20 0.42
10000
delete_big 0.00 0.00 0.00
0.00 1
delete_big 0.00 0.00 0.00
0.00 1
delete_big_many_keys 1065.00 0.01 0.00
0.01 128
delete_big_many_keys 1116.00 0.01 0.00
0.01 128
delete_commit 1.00 0.25 0.13 0.38
10000
delete_commit 1.00 0.33 0.13 0.46
10000
delete_key 2.00 0.20 0.19 0.39
10000
delete_key 2.00 0.29 0.14 0.43
10000
delete_range 8.00 0.00 0.00
0.00 12
delete_range 9.00 0.00 0.01
0.01 12
delete_rollback 2.00 0.24 0.11
0.35 100
delete_rollback 2.00 0.24 0.14
0.38 100
drop_index 2.00 0.00 0.00
0.00 8
drop_index 3.00 0.00 0.00
0.00 8
drop_table 0.00 0.00 0.00
0.00 28
drop_table 0.00 0.00 0.00
0.00 28
drop_table_when_MANY_tables 16.00 0.32 0.17 0.49
10000
drop_table_when_MANY_tables 17.00 0.49 0.20 0.69
10000
insert 90.00 8.15 6.68 14.83
350768
insert 92.00 8.19 6.69 14.88
350768
insert_autocommit 3.00 0.44 0.15 0.59
10000
insert_autocommit 3.00 0.45 0.22 0.67
10000
insert_commit 1.00 0.29 0.21 0.50
10000
insert_commit 1.00 0.30 0.20 0.50
10000
insert_duplicates 14.00 2.91 1.67 4.58
100000
insert_duplicates 14.00 3.33 1.62 4.95
100000
insert_key 523.00 5.69 2.11 7.80
100000
insert_key 532.00 6.30 2.25 8.55
100000
insert_many_fields 4.00 0.17 0.02 0.19
2000
insert_many_fields 4.00 0.23 0.04 0.27
2000
insert_rollback 1.00 0.33 0.21
0.54 100
insert_rollback 1.00 0.37 0.11
0.48 100
insert_select_1_key 4.00 0.00 0.00
0.00 1
insert_select_1_key 4.00 0.00 0.00
0.00 1
insert_select_2_keys 5.00 0.00 0.00
0.00 1
insert_select_2_keys 6.00 0.00 0.00
0.00 1
min_max 9.00 0.01 0.00
0.01 60
min_max 9.00 0.02 0.01
0.03 60
min_max_on_key 16.00 8.27 1.26 9.53
85000
min_max_on_key 18.00 10.03 1.30 11.33
85000
multiple_value_insert 2.00 0.12 0.01 0.13
100000
multiple_value_insert 3.00 0.15 0.01 0.16
100000
once_prepared_select 22.00 5.56 3.53 9.09
100000
once_prepared_select 22.00 6.23 3.74 9.97
100000
order_by_big 13.00 9.59 0.05
9.64 10
order_by_big 15.00 12.17 0.04
12.21 10
order_by_big_key 12.00 11.02 0.31
11.33 10
order_by_big_key 13.00 11.54 0.22
11.76 10
order_by_big_key2 11.00 10.05 0.04
10.09 10
order_by_big_key2 11.00 10.25 0.05
10.30 10
order_by_big_key_desc 13.00 11.37 0.17
11.54 10
order_by_big_key_desc 13.00 11.68 0.20
11.88 10
order_by_big_key_diff 11.00 9.73 0.09
9.82 10
order_by_big_key_diff 13.00 11.41 0.10
11.51 10
order_by_big_key_prefix 12.00 10.06 0.04
10.10 10
order_by_big_key_prefix 13.00 10.28 0.03
10.31 10
order_by_key2_diff 1.00 0.83 0.01
0.84 500
order_by_key2_diff 1.00 1.05 0.00
1.05 500
order_by_key_prefix 1.00 0.45 0.03
0.48 500
order_by_key_prefix 1.00 0.55 0.05
0.60 500
order_by_range 1.00 0.50 0.02
0.52 500
order_by_range 1.00 0.59 0.01
0.60 500
outer_join 23.00 0.00 0.00
0.00 10
outer_join 26.00 0.00 0.00
0.00 10
outer_join_found 21.00 0.00 0.00
0.00 10
outer_join_found 26.00 0.00 0.00
0.00 10
outer_join_not_found 21.00 0.02 0.00
0.02 500
outer_join_not_found 23.00 0.00 0.00
0.00 500
outer_join_on_key 13.00 0.00 0.00
0.00 10
outer_join_on_key 14.00 0.00 0.01
0.01 10
prepared_select 24.00 10.64 3.03 13.67
100000
prepared_select 26.00 10.86 3.20 14.06
100000
select_1_row 2.00 0.66 0.38 1.04
20000
select_1_row 2.00 0.68 0.33 1.01
20000
select_1_row_cache 2.00 0.47 0.36 0.83
20000
select_1_row_cache 3.00 0.51 0.26 0.77
20000
select_2_rows 2.00 0.51 0.26 0.77
20000
select_2_rows 3.00 0.54 0.30 0.84
20000
select_big 11.00 10.09 0.12
10.21 80
select_big 12.00 10.49 0.15
10.64 80
select_big_str 1.00 0.37 0.19 0.56
2000
select_big_str 1.00 0.46 0.13 0.59
2000
select_cache 55.00 1.03 0.18 1.21
10000
select_cache 59.00 0.94 0.16 1.10
10000
select_cache2 57.00 1.34 0.17 1.51
10000
select_cache2 61.00 1.21 0.20 1.41
10000
select_column+column 2.00 0.35 0.33 0.68
20000
select_column+column 2.00 0.45 0.31 0.76
20000
select_diff_key 0.00 0.05 0.01
0.06 500
select_diff_key 0.00 0.07 0.00
0.07 500
select_distinct 3.00 0.96 0.00
0.96 800
select_distinct 3.00 0.97 0.03
1.00 800
select_group 16.00 0.73 0.04 0.77
2911
select_group 18.00 0.65 0.07 0.72
2911
select_group_when_MANY_tables 7.00 0.30 0.20 0.50
10000
select_group_when_MANY_tables 8.00 0.35 0.15 0.50
10000
select_join 0.00 0.30 0.00
0.30 100
select_join 1.00 0.25 0.01
0.26 100
select_key 42.00 21.88 2.79 24.67
200000
select_key 44.00 22.09 3.11 25.20
200000
select_key2 49.00 21.36 3.34 24.70
200000
select_key2 49.00 22.73 3.19 25.92
200000
select_key2_return_key 44.00 19.47 3.27 22.74
200000
select_key2_return_key 48.00 19.89 3.56 23.45
200000
select_key2_return_prim 46.00 19.27 3.19 22.46
200000
select_key2_return_prim 55.00 24.64 3.86 28.50
200000
select_key_prefix 49.00 22.29 3.33 25.62
200000
select_key_prefix 53.00 29.17 3.25 32.42
200000
select_key_prefix_join 5.00 2.33 0.00
2.33 100
select_key_prefix_join 5.00 2.36 0.02
2.38 100
select_key_return_key 40.00 18.75 2.67 21.42
200000
select_key_return_key 40.00 19.96 2.82 22.78
200000
select_many_fields 16.00 8.97 0.18 9.15
2000
select_many_fields 16.00 9.11 0.17 9.28
2000
select_range 48.00 4.67 0.29
4.96 410
select_range 49.00 4.59 0.23
4.82 410
select_range_key2 7.00 2.61 0.34 2.95
25010
select_range_key2 7.00 2.69 0.28 2.97
25010
select_range_prefix 7.00 2.14 0.28 2.42
25010
select_range_prefix 8.00 2.54 0.35 2.89
25010
select_simple 1.00 0.29 0.35 0.64
20000
select_simple 1.00 0.31 0.40 0.71
20000
select_simple_cache 2.00 0.34 0.30 0.64
20000
select_simple_cache 2.00 0.40 0.36 0.76
20000
select_simple_join 0.00 0.34 0.02
0.36 500
select_simple_join 1.00 0.35 0.06
0.41 500
update_autocommit 3.00 0.36 0.15 0.51
10000
update_autocommit 3.00 0.37 0.12 0.49
10000
update_big 45.00 0.01 0.00
0.01 10
update_big 46.00 0.00 0.00
0.00 10
update_commit 1.00 0.27 0.18 0.45
10000
update_commit 1.00 0.33 0.13 0.46
10000
update_of_key 30.00 1.45 0.96 2.41
50000
update_of_key 31.00 1.67 0.86 2.53
50000
update_of_key_big 38.00 0.02 0.00
0.02 501
update_of_key_big 44.00 0.00 0.03
0.03 501
update_of_primary_key_many_keys 1196.00 0.03 0.01
0.04 256
update_of_primary_key_many_keys 1236.00 0.03 0.00
0.03 256
update_rollback 1.00 0.30 0.22
0.52 100
update_rollback 1.00 0.33 0.17
0.50 100
update_with_key 93.00 7.86 5.44 13.30
300000
update_with_key 94.00 8.64 5.53 14.17
300000
update_with_key_prefix 36.00 4.86 2.18 7.04
100000
update_with_key_prefix 36.00 5.17 2.18 7.35
100000
wisc_benchmark 2.00 1.77 0.11
1.88 114
wisc_benchmark 3.00 1.72 0.11
1.83 114
TOTALS 5295.00 324.05 57.00 381.05
2974250
TOTALS 5448.00 314.80 57.91 372.71
2974250
_______________________________________________
Mailing list: https://launchpad.net/~drizzle-benchmark
Post to : [email protected]
Unsubscribe : https://launchpad.net/~drizzle-benchmark
More help : https://help.launchpad.net/ListHelp
_______________________________________________
Mailing list: https://launchpad.net/~drizzle-benchmark
Post to : [email protected]
Unsubscribe : https://launchpad.net/~drizzle-benchmark
More help : https://help.launchpad.net/ListHelp
_______________________________________________
Mailing list: https://launchpad.net/~drizzle-benchmark
Post to : [email protected]
Unsubscribe : https://launchpad.net/~drizzle-benchmark
More help : https://help.launchpad.net/ListHelp
