On Tue, Feb 12, 2019 at 4:23 AM Jung, Jinho <jinho.j...@gatech.edu> wrote:
> > Hello, > > We are developing a tool called sqlfuzz for automatically finding > performance regressions in PostgreSQL. sqlfuzz performs mutational fuzzing > to generate SQL queries that take more time to execute on the latest > version of PostgreSQL compared to prior versions. We hope that these > queries would help further increase the utility of the regression test > suite. > > We would greatly appreciate feedback from the community regarding the > queries found by the tool so far. We have already incorporated prior > feedback from the community in the latest version of sqlfuzz. > This approach doesn't seem very exciting to me as-is, because optimization is a very pragmatic endeavor. We make decisions all the time that might make some queries better and others worse. If the queries that get better are natural/common ones, and the ones that get worse are weird/uncommon ones (like generated by a fuzzer), then making that change is an improvement even if there are some performance (as opposed to correctness) regressions. I would be more interested in investigating some of these if the report would: 1) include the exact commit in which the regression was introduced (i.e. automate "git bisect"). 2) verify that the regression still exists in the dev HEAD and report which commit it was verified in (since HEAD changes frequently). 3) report which queries (if any) in your corpus were made better by the same commit which made the victim query worse. Cheers, Jeff >
