Hi all,
I wasn't sure if I should add this to the QA discussion, but it seemed
worthy of its own thread.
During the "future of the staff client" meeting, I advocated for
bringing in a consultant to do a software performance analysis for
Evergreen to help us identify where the critical bottlenecks are in the
system in the hopes that we could then identify the areas that need to
be worked on to improve performance. At the time, I didn't have any
concrete suggestions on finding a consultant who could take on this
project, but I have since done some more investigation and have a couple
of leads, the most promising of which is an individual local to
Massachusetts who previously worked for many years at Stratus
Technologies where he was involved in all levels of performance
analysis. He now teaches graduate-level courses on performance
evaluation and also does contract work.
Now that I actually have concrete leads, I would like to get the ball
rolling, provided there is support from the larger community. I'm not
quite sure how this might fit in with ESI's planned QA efforts or with
the possibility of bringing in a firm like OmniTI as Dan suggested, but
my reading into these QA e-mails is that the focus would be on testing
new commits. I fully support improved testing for new code, but my main
concern now is testing what is already there since improving existing
performance issues is a high priority for us. I think there is
opportunity for these efforts to support one another.
As I mentioned during the meeting, MassLNC is willing to contribute
financially to bringing on a consultant, but I think it's important that
this effort is a community project for several reasons. If we do bring a
consultant, in order to be successful, it is imperative that they work
with the community of sys admins and developers who already have insight
on where the underlying issues might be. Also, if we don't have buy-in
from the community, then it is likely that any work a consultant does
will be largely ignored. My hope is that any work done by a
consultant(s) could be a real starting point in resolving any of the
critical bottlenecks that were identified in a performance evaluation.
So I guess my question is whether there is any support from the
community to proceed with finding someone to do a software performance
analysis?
If the answer is yes, then there are at least two things that need to be
done before going any further.
1. We need to define the areas that need analysis. Bill Erickson had
suggested during the meeting that we might find a lot of areas of
agreement if we worked on our own list of ideas before contracting with
anyone. The consultant I spoke with also suggested that we identify the
areas where the pain is being most felt. If there is support for moving
forward, perhaps we can begin to build this list on the wiki. As a side
note, the consultant also recommended pulling together a group of
stakeholders to oversee this process. I initially was thinking this
group could just be the existing developer community, but, with the
recent flurry of discussion over QA and performance, I'm now wondering
if it might be worthwhile to have a QA group that focuses on
coordinating these efforts and identifying next steps.
2. We also would need to identify an environment to do the performance
testing. In speaking with the performance consultant, the ideal testing
environment would be a server that has a tool that drives transactions.
Is that something that testing.esilibrary.com has? If not, is it
something that could be built. Can Cucumber help us with that?
I appreciate any thoughts you all have on this subject.
Kathy
--
Kathy Lussier
Project Coordinator
Massachusetts Library Network Cooperative
(508) 343-0128
[email protected]
Twitter: http://www.twitter.com/kmlussier
