First off, thanks for the good-will in taking the time to ask.
My biggest concern in adopting it as a codebase would be ensuring that
it isn't another codebase dropped into contrib/ and subsequently
ignored. How do you plan to avoid this? Who do you see maintaining and
running these tests?
Some more targeted implementation observations/questions -
* Do you plan to update the scripts to work with Apache Accumulo instead
of CDH specific artifacts? e.g. [1]
* For the MapReduce job specifically, why did you write your own and not
use an existing "vetted" job like Continuous Ingest? Is there something
that the included M/R job does which is not already contained by our CI
ingest and verify jobs?
* It looks like the current script only works for 1.4 to 1.6? Do you
plan to support 1.5->1.6, 1.5->1.7, 1.6->1.7? How do you envision this
adoption occurring?
* As far as exercising internal Accumulo implementation, I think you
have the basics covered. What about some more tricky things over the
metadata table (clone, import, export, merge, split table)? How might
additional functionality be added in a way that can be automatically tested?
* It seems like you have also targeted a physical set of nodes. Have you
considered actually using some virtualization platform (e.g. vagrant) to
fully automate upgrade-testing? If there is a way that a user can spin
up a few VMs to do the testing, the barrier to entry is much lower (and
likely more foolproof) than requiring the user to set up the environment.
Thanks, Sean.
- Josh
[1]
https://github.com/cloudera/accumulo-upgrade-test/blob/master/upgrade-test.sh#L35
Sean Busbey wrote:
Hi Accumulo!
Cloudera has recently made public some code used for doing correctness
testing for Accumulo installations across upgrades[1]. The project contains
simple data load and verification tools as well as a rudimentary upgrade
test automation script.
We would like to donate this code to the ASF and use it as a starting place
for a contrib repository focused on testing Accumulo across versions
generally. If the PMC is interested, I'd be happy to take point on the
necessary paperwork for IP clearance[2] following an appropriate vote.
Though simple, this upgrade testing has already found issues around 1.4 ->
1.5 -> 1.6 and 1.4 ->1.6 upgrades in the past. There are some obvious
points of clean up that need to happen (vendor specific references in docs,
artifacts, and packages), and I'd be happy to take point on that work as
well. I think it would make a good foundational layer for our future needs
and would give us a place where we could revitalize the work from
ACCUMULO-2145.
Thoughts?
[1]: https://github.com/cloudera/accumulo-upgrade-test/
[2]: http://incubator.apache.org/ip-clearance/
