Paul Poulosky created STORM-1508:
------------------------------------
Summary: Investigate creating integration test framework
Key: STORM-1508
URL: https://issues.apache.org/jira/browse/STORM-1508
Project: Apache Storm
Issue Type: Story
Reporter: Paul Poulosky
This is related to STORM-1179, and an offline discussion.
It would be helpful if we could have an integration test framework that could
deploy storm via standard tools and run true integration tests. This would be
useful for test scenarios that cannot be tested with LocalCluster.
The test framework would need to
1) node / VM allocation
2) deployment
3) modify configuration
4) run tests / report results
Ducktape (https://github.com/confluentinc/ducktape/wiki) has been suggested as
a common framework we could use to perform this type of testing.
>From offline discussion:
Yes we too have similar tests based on internal Yahoo testing. Really it comes
down to a few operations that the test framework needs to be able to do 1)
node/VM allocation 2) deployment 3) modify configuration/processes on running
nodes 4) run actual tests.
The first three are often specific to the setup. Is it open stack? Amazon
EC2? Are we using ssh to go between nodes, or is it something else? All of
this is something that we should be able to accomplish with an abstraction
layer. ducktape appears to have a fairly decent abstraction layer for the
first even allowing for a static JSON config for the hosts. The only other
requirement seems to be ssh access between the nodes so it can run various
python scripts. As we are developing these tests we probably want to be sure
that we have a pluggable abstraction for the other layers as well. That way if
I am using RPMs and Chef to do a deployment and be sure my RPM packaging is
correct I can do that, and if someone else just wants to take the TGZ and scp
it everywhere that works too.
I also hope that we can limit the score of these tests. The current Yahoo
integration tests take several hours to run because we are constantly trying to
reconfigure the cluster to test different setups. Ideally we can test failure
scenarios and basic integration with other tools, but anything that can be done
on a LocalCluster should probably be tested there, to reduce the total running
time and debugging complexity.
- Bobby
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
