> On 14 Mar 2018, at 19:14, Mark Wong <m...@2ndquadrant.com> wrote: > > Hi, > > I have some additional comments to a couple areas... > >> On Wed, Mar 14, 2018 at 05:33:00PM -0400, Dave Page wrote: >>> On Tue, Mar 13, 2018 at 11:31 PM, Hongyuan Ma <cs_maleica...@163.com> wrote: >>> At the same time I hope you can help me understand the functional >>> requirements of this project more clearly. Here are some of my thoughts >>> on PerfFarm: >>> >>> - I see this comment in the client folder (In line 15 of the "pgperffarm >>> \ client \ benchmarks \ pgbench.py" file): >>> ''' >>> # TODO allow running custom scripts, not just the default >>> ''' >>> Will PerfFarm have many test items so that the test item search function >>> or even the test item sort function is expected to be provided? >>> >> >> I don't know - Tomas or Mark are probably the best folks to ask about that. >> I spent some time on the initial web app work, but then ran out of spare >> cycles. > > Yes, there is potential for the number of tests to change over time. > (Maybe a test is less relevant, maybe there is a more useful test down > he road.) > > I'm haven't thought too much about sorting and searching functions. But > right now there is just pgbench with 6 basic configurations: read/write, > read-only, and a low, medium, and high data set size for each of those. > >>> - What value will be used to determine if a machine's performance is >>> improving or declining? >>> >> >> Good question - I think that needs discussion, as it's not necessarily >> clear-cut when you think that performance may vary at different numbers of >> clients. > > My thought here is that each test should provide a single metric, > defined by the test itself, so expect this to be define in the test > results sent to the web site. > > I do expect that off of the test results should be archived somehow and > retrievable. But the idea I have is to have a generic high level view > that all of the different tests can share. > >>> - I see BuildFarm assigning an animal name to each registered machine. Will >>> PerfFarm also have this interesting feature? >>> >> >> It was going to, but I forget what we decided on for a naming scheme! >> Another discussion item I think - in the code, we can just use "Name" or >> similar. > > Since the buildfarm is composed of animals, I thought plants would be a > complimentary scheme? I'm also not sure if this was discussed > previously...
For some reason I was thinking fish!
