+1 I think we should experiment with this. If it can serve us well, and we all agree, then we should consider adopting it.
From what I see: * Icense is compatible * fairly well documented. What potentially worries me: * Maintenance. Can we count on updates? Is the ducktape community open to accepting patches from us? If so, what would be the turnaround time for acceptance? -Taylor > On Nov 5, 2015, at 8:54 PM, Harsha <[email protected]> wrote: > > The reason for suggesting ducktape not just used in apache kafka but > also its getting security services integration like kdc and already has > zookeeper. This framework can work with vagrant vms or amazon ec2 etc.. > >> On Thu, Nov 5, 2015, at 11:49 AM, Hugo Da Cruz Louro wrote: >> Great, will make this a priority. Created a >> JIRA<https://issues.apache.org/jira/browse/STORM-1179> ticket and >> assigned it to me. >> >> https://issues.apache.org/jira/browse/STORM-1179 >> >> >> On Nov 5, 2015, at 11:40 AM, Bobby Evans >> <[email protected]<mailto:[email protected]>> wrote: >> >> Hugo, >> >> I would love to see that happen. It has been on my list for a while, but >> I have never found the time to do it. I personally am +1 on this, >> hopefully we can do this quickly in preparation for a 0.11.0 release. >> >> - Bobby >> >> >> >> On Thursday, November 5, 2015 1:37 PM, Hugo Da Cruz Louro >> <[email protected]<mailto:[email protected]>> wrote: >> >> >> I a agree with these three levels of testing, and that at the very least >> we should keep unit tests separated from the system/integration tests. >> One huge advantage would be to quickly run unit tests that hopefully are >> more predictable, and thus avoid intermittent test fails. >> >> Bobby just mentioned but I had already in mind that it would be useful to >> create different maven profiles for integration and unit tests. I have >> done something similar in a different project and it works really well. >> If we agree that it is something we want to implement here, I will create >> a JIRA ticket for this and get it done. Please let me know. >> >> Thanks, >> Hugo >> >>> On Nov 5, 2015, at 11:23 AM, Bobby Evans >>> <[email protected]<mailto:[email protected]>> wrote: >>> >>> I totally agree. Too many of our "unit" tests are integration tests and >>> end up spinning up an entire local-mode cluster. >>> I personally would like to see three levels of testing. >>> 1) true unit tests. They only touch the code under test and very little >>> else. The should be what you get when you run mvn test2) integration >>> tests. These should spin up local mode clusters and modify configs/etc to >>> get a decent set of more white box tests. The should run as a part of >>> trivis-ci, and probably should run by enabling a special profile.3) Sanity >>> Integration Tests. ducktape looks like a great fit here. I would love to >>> see us spin up an few different scenarios for testing, with/without >>> security. Talking to Kafka, Hadoop(HBase, HDFS, Hive), redis, >>> elastasearch, etc. >>> These would be run frequently but not necessarily a part of CI initially. >>> - Bobby >>> >>> >>> On Wednesday, November 4, 2015 7:21 PM, Harsha >>> <[email protected]<mailto:[email protected]>> wrote: >>> >>> >>> Hi All, >>> As community we are growing and adding new and exciting >>> features to Storm and also we've ever growing connector which >>> only helps in storm adoption. One thing we've severely lacking >>> is system tests. There are unit tests which acts as >>> integration tests to storm-core but there are no system wide >>> integration tests that can spin up kafka nodes and storm , >>> hbase and run a topology that can make sure the data is >>> getting into hbase. >>> We at Hortonworks use our test topologies run some of these >>> tests but this integration code to spin up vms or nodes is hard >>> to share. In apache kafka we are using ducktape to write system >>> tests so far its working out good. If there are any other >>> frameworks you've in mind we can definitely take a look. But as >>> a community we need start looking at >>> https://github.com/confluentinc/ducktape or similar frameworks >>> to start writing systems tests. Ducktape makes it easy to run >>> in a vm or any other infrastructure. Appreciate any feedback >>> on this. >>> >>> Thanks, >>> Harsha >> >> >>
