Thanks very much to Patrick for getting sql-bench setup up and running
again, we have a basic run now automated through Hudson. Attached
(build-1113.txt) is typical output from a sql-bench run. I'd like to get
input on what we want to see in an email after each run. Diff'ing the
results with a previous run presents too much info since the times are
so small. A few ideas:
* Start with a compare of just theTOTALS line with previous runs
* Take the last section "Totals per operation" to the end and sort
it with a number previous runs so you would see something like the
second attachment (which only has data for 2 runs)
* Create a master file to compare against, if any times go over by a
certain percentage, raise the red flag
Other suggestions/ideas??
-Lee
Benchmark DBD suite: 2.15
Date of test: 2009-08-11 12:22:57
Running tests on: Linux 2.6.28-11-server x86_64
Arguments: --connect-options=port=9306
Comments:
Limits from:
Server version: Drizzle 2009.08.1113
Optimization: None
Hardware:
alter-table: Total time: 29 wallclock secs ( 0.03 usr 0.03 sys + 0.00 cusr
0.00 csys = 0.06 CPU)
ATIS: Total time: 16 wallclock secs ( 4.74 usr 0.28 sys + 0.00 cusr 0.00
csys = 5.02 CPU)
big-tables: Total time: 20 wallclock secs ( 9.14 usr 0.20 sys + 0.00 cusr
0.00 csys = 9.34 CPU)
connect: Total time: 19 wallclock secs ( 6.26 usr 3.80 sys + 0.00 cusr 0.00
csys = 10.06 CPU)
create: Total time: 82 wallclock secs ( 5.18 usr 2.16 sys + 0.00 cusr 0.00
csys = 7.34 CPU)
insert: Total time: 3734 wallclock secs (270.70 usr 45.50 sys + 0.00 cusr
0.00 csys = 316.20 CPU)
select: Total time: 319 wallclock secs (28.24 usr 3.05 sys + 0.00 cusr 0.00
csys = 31.29 CPU)
transactions: Total time: 16 wallclock secs ( 2.89 usr 1.40 sys + 0.00 cusr
0.00 csys = 4.29 CPU)
wisconsin: Total time: 12 wallclock secs ( 2.46 usr 0.68 sys + 0.00 cusr
0.00 csys = 3.14 CPU)
All 9 test executed successfully
Totals per operation:
Operation seconds usr sys cpu tests
alter_table_add 13.00 0.01 0.00 0.01 100
alter_table_drop 12.00 0.00 0.00 0.00 91
connect 2.00 0.84 0.36 1.20 2000
connect+select_1_row 2.00 1.01 0.56 1.57 2000
connect+select_simple 2.00 1.22 0.72 1.94 2000
count 16.00 0.01 0.00 0.01 100
count_distinct 5.00 0.11 0.00 0.11 1000
count_distinct_2 12.00 0.21 0.02 0.23 1000
count_distinct_big 20.00 4.24 0.00 4.24 120
count_distinct_group 13.00 0.72 0.05 0.77 1000
count_distinct_group_on_key 10.00 0.21 0.01 0.22 1000
count_distinct_group_on_key_parts 12.00 0.77 0.05 0.82 1000
count_distinct_key_prefix 4.00 0.15 0.02 0.17 1000
count_group_on_key_parts 8.00 0.35 0.04 0.39 1000
count_on_key 72.00 5.50 0.95 6.45 50100
create+drop 19.00 1.43 0.58 2.01 10000
create_MANY_tables 17.00 1.26 0.44 1.70 10000
create_index 2.00 0.00 0.00 0.00 8
create_key+drop 22.00 1.70 0.74 2.44 10000
create_table 0.00 0.00 0.00 0.00 31
delete_all_many_keys 1065.00 0.01 0.00 0.01 1
delete_autocommit 3.00 0.19 0.14 0.33 10000
delete_big 0.00 0.00 0.00 0.00 1
delete_big_many_keys 1065.00 0.01 0.00 0.01 128
delete_commit 1.00 0.33 0.13 0.46 10000
delete_key 2.00 0.29 0.14 0.43 10000
delete_range 8.00 0.00 0.00 0.00 12
delete_rollback 2.00 0.24 0.14 0.38 100
drop_index 2.00 0.00 0.00 0.00 8
drop_table 0.00 0.00 0.00 0.00 28
drop_table_when_MANY_tables 17.00 0.49 0.20 0.69 10000
insert 92.00 8.19 6.69 14.88 350768
insert_autocommit 3.00 0.45 0.22 0.67 10000
insert_commit 1.00 0.29 0.21 0.50 10000
insert_duplicates 14.00 2.91 1.67 4.58 100000
insert_key 523.00 5.69 2.11 7.80 100000
insert_many_fields 4.00 0.17 0.02 0.19 2000
insert_rollback 1.00 0.37 0.11 0.48 100
insert_select_1_key 4.00 0.00 0.00 0.00 1
insert_select_2_keys 6.00 0.00 0.00 0.00 1
min_max 9.00 0.01 0.00 0.01 60
min_max_on_key 18.00 10.03 1.30 11.33 85000
multiple_value_insert 2.00 0.12 0.01 0.13 100000
once_prepared_select 22.00 5.56 3.53 9.09 100000
order_by_big 15.00 12.17 0.04 12.21 10
order_by_big_key 13.00 11.54 0.22 11.76 10
order_by_big_key2 11.00 10.25 0.05 10.30 10
order_by_big_key_desc 13.00 11.68 0.20 11.88 10
order_by_big_key_diff 13.00 11.41 0.10 11.51 10
order_by_big_key_prefix 13.00 10.28 0.03 10.31 10
order_by_key2_diff 1.00 0.83 0.01 0.84 500
order_by_key_prefix 1.00 0.45 0.03 0.48 500
order_by_range 1.00 0.50 0.02 0.52 500
outer_join 23.00 0.00 0.00 0.00 10
outer_join_found 21.00 0.00 0.00 0.00 10
outer_join_not_found 21.00 0.02 0.00 0.02 500
outer_join_on_key 13.00 0.00 0.00 0.00 10
prepared_select 26.00 10.86 3.20 14.06 100000
select_1_row 2.00 0.68 0.33 1.01 20000
select_1_row_cache 2.00 0.47 0.36 0.83 20000
select_2_rows 3.00 0.54 0.30 0.84 20000
select_big 12.00 10.49 0.15 10.64 80
select_big_str 1.00 0.46 0.13 0.59 2000
select_cache 55.00 1.03 0.18 1.21 10000
select_cache2 61.00 1.21 0.20 1.41 10000
select_column+column 2.00 0.35 0.33 0.68 20000
select_diff_key 0.00 0.05 0.01 0.06 500
select_distinct 3.00 0.96 0.00 0.96 800
select_group 18.00 0.65 0.07 0.72 2911
select_group_when_MANY_tables 7.00 0.30 0.20 0.50 10000
select_join 1.00 0.25 0.01 0.26 100
select_key 44.00 22.09 3.11 25.20 200000
select_key2 49.00 22.73 3.19 25.92 200000
select_key2_return_key 44.00 19.47 3.27 22.74 200000
select_key2_return_prim 46.00 19.27 3.19 22.46 200000
select_key_prefix 53.00 29.17 3.25 32.42 200000
select_key_prefix_join 5.00 2.33 0.00 2.33 100
select_key_return_key 40.00 19.96 2.82 22.78 200000
select_many_fields 16.00 8.97 0.18 9.15 2000
select_range 49.00 4.59 0.23 4.82 410
select_range_key2 7.00 2.61 0.34 2.95 25010
select_range_prefix 7.00 2.14 0.28 2.42 25010
select_simple 1.00 0.29 0.35 0.64 20000
select_simple_cache 2.00 0.40 0.36 0.76 20000
select_simple_join 0.00 0.34 0.02 0.36 500
update_autocommit 3.00 0.36 0.15 0.51 10000
update_big 45.00 0.01 0.00 0.01 10
update_commit 1.00 0.33 0.13 0.46 10000
update_of_key 31.00 1.67 0.86 2.53 50000
update_of_key_big 44.00 0.00 0.03 0.03 501
update_of_primary_key_many_keys 1196.00 0.03 0.01 0.04 256
update_rollback 1.00 0.33 0.17 0.50 100
update_with_key 93.00 7.86 5.44 13.30 300000
update_with_key_prefix 36.00 4.86 2.18 7.04 100000
wisc_benchmark 3.00 1.72 0.11 1.83 114
TOTALS 5295.00 324.05 57.00 381.05 2974250
Operation seconds usr sys cpu tests
alter_table_add 13.00 0.01 0.00 0.01 100
alter_table_add 15.00 0.00 0.01 0.01 100
alter_table_drop 12.00 0.00 0.00 0.00 91
alter_table_drop 13.00 0.02 0.01 0.03 91
connect 2.00 0.59 0.33 0.92 2000
connect 2.00 0.84 0.36 1.20 2000
connect+select_1_row 2.00 1.01 0.56 1.57 2000
connect+select_1_row 2.00 1.04 0.55 1.59 2000
connect+select_simple 2.00 1.17 0.66 1.83 2000
connect+select_simple 2.00 1.22 0.72 1.94 2000
count 16.00 0.01 0.00 0.01 100
count 16.00 0.04 0.00 0.04 100
count_distinct 5.00 0.11 0.00 0.11 1000
count_distinct 7.00 0.05 0.02 0.07 1000
count_distinct_2 12.00 0.21 0.02 0.23 1000
count_distinct_2 14.00 0.34 0.06 0.40 1000
count_distinct_big 20.00 4.24 0.00 4.24 120
count_distinct_big 21.00 4.22 0.02 4.24 120
count_distinct_group 11.00 0.28 0.04 0.32 1000
count_distinct_group 13.00 0.72 0.05 0.77 1000
count_distinct_group_on_key 9.00 0.16 0.06 0.22 1000
count_distinct_group_on_key 10.00 0.21 0.01 0.22 1000
count_distinct_group_on_key_parts 10.00 0.00 0.00 0.00 1000
count_distinct_group_on_key_parts 12.00 0.77 0.05 0.82 1000
count_distinct_key_prefix 4.00 0.11 0.03 0.14 1000
count_distinct_key_prefix 4.00 0.15 0.02 0.17 1000
count_group_on_key_parts 8.00 0.35 0.04 0.39 1000
count_group_on_key_parts 8.00 0.49 0.02 0.51 1000
count_on_key 72.00 5.50 0.95 6.45 50100
count_on_key 73.00 5.95 0.90 6.85 50100
create+drop 19.00 1.39 0.64 2.03 10000
create+drop 19.00 1.43 0.58 2.01 10000
create_MANY_tables 15.00 1.15 0.42 1.57 10000
create_MANY_tables 17.00 1.26 0.44 1.70 10000
create_index 2.00 0.00 0.00 0.00 8
create_index 2.00 0.00 0.00 0.00 8
create_key+drop 22.00 1.65 0.56 2.21 10000
create_key+drop 22.00 1.70 0.74 2.44 10000
create_table 0.00 0.00 0.00 0.00 31
create_table 0.00 0.01 0.01 0.02 31
delete_all_many_keys 1065.00 0.01 0.00 0.01 1
delete_all_many_keys 1117.00 0.01 0.00 0.01 1
delete_autocommit 3.00 0.19 0.14 0.33 10000
delete_autocommit 3.00 0.22 0.20 0.42 10000
delete_big 0.00 0.00 0.00 0.00 1
delete_big 0.00 0.00 0.00 0.00 1
delete_big_many_keys 1065.00 0.01 0.00 0.01 128
delete_big_many_keys 1116.00 0.01 0.00 0.01 128
delete_commit 1.00 0.25 0.13 0.38 10000
delete_commit 1.00 0.33 0.13 0.46 10000
delete_key 2.00 0.20 0.19 0.39 10000
delete_key 2.00 0.29 0.14 0.43 10000
delete_range 8.00 0.00 0.00 0.00 12
delete_range 9.00 0.00 0.01 0.01 12
delete_rollback 2.00 0.24 0.11 0.35 100
delete_rollback 2.00 0.24 0.14 0.38 100
drop_index 2.00 0.00 0.00 0.00 8
drop_index 3.00 0.00 0.00 0.00 8
drop_table 0.00 0.00 0.00 0.00 28
drop_table 0.00 0.00 0.00 0.00 28
drop_table_when_MANY_tables 16.00 0.32 0.17 0.49 10000
drop_table_when_MANY_tables 17.00 0.49 0.20 0.69 10000
insert 90.00 8.15 6.68 14.83 350768
insert 92.00 8.19 6.69 14.88 350768
insert_autocommit 3.00 0.44 0.15 0.59 10000
insert_autocommit 3.00 0.45 0.22 0.67 10000
insert_commit 1.00 0.29 0.21 0.50 10000
insert_commit 1.00 0.30 0.20 0.50 10000
insert_duplicates 14.00 2.91 1.67 4.58 100000
insert_duplicates 14.00 3.33 1.62 4.95 100000
insert_key 523.00 5.69 2.11 7.80 100000
insert_key 532.00 6.30 2.25 8.55 100000
insert_many_fields 4.00 0.17 0.02 0.19 2000
insert_many_fields 4.00 0.23 0.04 0.27 2000
insert_rollback 1.00 0.33 0.21 0.54 100
insert_rollback 1.00 0.37 0.11 0.48 100
insert_select_1_key 4.00 0.00 0.00 0.00 1
insert_select_1_key 4.00 0.00 0.00 0.00 1
insert_select_2_keys 5.00 0.00 0.00 0.00 1
insert_select_2_keys 6.00 0.00 0.00 0.00 1
min_max 9.00 0.01 0.00 0.01 60
min_max 9.00 0.02 0.01 0.03 60
min_max_on_key 16.00 8.27 1.26 9.53 85000
min_max_on_key 18.00 10.03 1.30 11.33 85000
multiple_value_insert 2.00 0.12 0.01 0.13 100000
multiple_value_insert 3.00 0.15 0.01 0.16 100000
once_prepared_select 22.00 5.56 3.53 9.09 100000
once_prepared_select 22.00 6.23 3.74 9.97 100000
order_by_big 13.00 9.59 0.05 9.64 10
order_by_big 15.00 12.17 0.04 12.21 10
order_by_big_key 12.00 11.02 0.31 11.33 10
order_by_big_key 13.00 11.54 0.22 11.76 10
order_by_big_key2 11.00 10.05 0.04 10.09 10
order_by_big_key2 11.00 10.25 0.05 10.30 10
order_by_big_key_desc 13.00 11.37 0.17 11.54 10
order_by_big_key_desc 13.00 11.68 0.20 11.88 10
order_by_big_key_diff 11.00 9.73 0.09 9.82 10
order_by_big_key_diff 13.00 11.41 0.10 11.51 10
order_by_big_key_prefix 12.00 10.06 0.04 10.10 10
order_by_big_key_prefix 13.00 10.28 0.03 10.31 10
order_by_key2_diff 1.00 0.83 0.01 0.84 500
order_by_key2_diff 1.00 1.05 0.00 1.05 500
order_by_key_prefix 1.00 0.45 0.03 0.48 500
order_by_key_prefix 1.00 0.55 0.05 0.60 500
order_by_range 1.00 0.50 0.02 0.52 500
order_by_range 1.00 0.59 0.01 0.60 500
outer_join 23.00 0.00 0.00 0.00 10
outer_join 26.00 0.00 0.00 0.00 10
outer_join_found 21.00 0.00 0.00 0.00 10
outer_join_found 26.00 0.00 0.00 0.00 10
outer_join_not_found 21.00 0.02 0.00 0.02 500
outer_join_not_found 23.00 0.00 0.00 0.00 500
outer_join_on_key 13.00 0.00 0.00 0.00 10
outer_join_on_key 14.00 0.00 0.01 0.01 10
prepared_select 24.00 10.64 3.03 13.67 100000
prepared_select 26.00 10.86 3.20 14.06 100000
select_1_row 2.00 0.66 0.38 1.04 20000
select_1_row 2.00 0.68 0.33 1.01 20000
select_1_row_cache 2.00 0.47 0.36 0.83 20000
select_1_row_cache 3.00 0.51 0.26 0.77 20000
select_2_rows 2.00 0.51 0.26 0.77 20000
select_2_rows 3.00 0.54 0.30 0.84 20000
select_big 11.00 10.09 0.12 10.21 80
select_big 12.00 10.49 0.15 10.64 80
select_big_str 1.00 0.37 0.19 0.56 2000
select_big_str 1.00 0.46 0.13 0.59 2000
select_cache 55.00 1.03 0.18 1.21 10000
select_cache 59.00 0.94 0.16 1.10 10000
select_cache2 57.00 1.34 0.17 1.51 10000
select_cache2 61.00 1.21 0.20 1.41 10000
select_column+column 2.00 0.35 0.33 0.68 20000
select_column+column 2.00 0.45 0.31 0.76 20000
select_diff_key 0.00 0.05 0.01 0.06 500
select_diff_key 0.00 0.07 0.00 0.07 500
select_distinct 3.00 0.96 0.00 0.96 800
select_distinct 3.00 0.97 0.03 1.00 800
select_group 16.00 0.73 0.04 0.77 2911
select_group 18.00 0.65 0.07 0.72 2911
select_group_when_MANY_tables 7.00 0.30 0.20 0.50 10000
select_group_when_MANY_tables 8.00 0.35 0.15 0.50 10000
select_join 0.00 0.30 0.00 0.30 100
select_join 1.00 0.25 0.01 0.26 100
select_key 42.00 21.88 2.79 24.67 200000
select_key 44.00 22.09 3.11 25.20 200000
select_key2 49.00 21.36 3.34 24.70 200000
select_key2 49.00 22.73 3.19 25.92 200000
select_key2_return_key 44.00 19.47 3.27 22.74 200000
select_key2_return_key 48.00 19.89 3.56 23.45 200000
select_key2_return_prim 46.00 19.27 3.19 22.46 200000
select_key2_return_prim 55.00 24.64 3.86 28.50 200000
select_key_prefix 49.00 22.29 3.33 25.62 200000
select_key_prefix 53.00 29.17 3.25 32.42 200000
select_key_prefix_join 5.00 2.33 0.00 2.33 100
select_key_prefix_join 5.00 2.36 0.02 2.38 100
select_key_return_key 40.00 18.75 2.67 21.42 200000
select_key_return_key 40.00 19.96 2.82 22.78 200000
select_many_fields 16.00 8.97 0.18 9.15 2000
select_many_fields 16.00 9.11 0.17 9.28 2000
select_range 48.00 4.67 0.29 4.96 410
select_range 49.00 4.59 0.23 4.82 410
select_range_key2 7.00 2.61 0.34 2.95 25010
select_range_key2 7.00 2.69 0.28 2.97 25010
select_range_prefix 7.00 2.14 0.28 2.42 25010
select_range_prefix 8.00 2.54 0.35 2.89 25010
select_simple 1.00 0.29 0.35 0.64 20000
select_simple 1.00 0.31 0.40 0.71 20000
select_simple_cache 2.00 0.34 0.30 0.64 20000
select_simple_cache 2.00 0.40 0.36 0.76 20000
select_simple_join 0.00 0.34 0.02 0.36 500
select_simple_join 1.00 0.35 0.06 0.41 500
update_autocommit 3.00 0.36 0.15 0.51 10000
update_autocommit 3.00 0.37 0.12 0.49 10000
update_big 45.00 0.01 0.00 0.01 10
update_big 46.00 0.00 0.00 0.00 10
update_commit 1.00 0.27 0.18 0.45 10000
update_commit 1.00 0.33 0.13 0.46 10000
update_of_key 30.00 1.45 0.96 2.41 50000
update_of_key 31.00 1.67 0.86 2.53 50000
update_of_key_big 38.00 0.02 0.00 0.02 501
update_of_key_big 44.00 0.00 0.03 0.03 501
update_of_primary_key_many_keys 1196.00 0.03 0.01 0.04 256
update_of_primary_key_many_keys 1236.00 0.03 0.00 0.03 256
update_rollback 1.00 0.30 0.22 0.52 100
update_rollback 1.00 0.33 0.17 0.50 100
update_with_key 93.00 7.86 5.44 13.30 300000
update_with_key 94.00 8.64 5.53 14.17 300000
update_with_key_prefix 36.00 4.86 2.18 7.04 100000
update_with_key_prefix 36.00 5.17 2.18 7.35 100000
wisc_benchmark 2.00 1.77 0.11 1.88 114
wisc_benchmark 3.00 1.72 0.11 1.83 114
TOTALS 5295.00 324.05 57.00 381.05 2974250
TOTALS 5448.00 314.80 57.91 372.71 2974250
_______________________________________________
Mailing list: https://launchpad.net/~drizzle-benchmark
Post to : [email protected]
Unsubscribe : https://launchpad.net/~drizzle-benchmark
More help : https://help.launchpad.net/ListHelp
