Thanks for the response John. I should point out that for streams app testing, we generally do use the TopologyTestDriver and it is much appreciated. We use EmbeddedKafkaCluster more for testing code that uses the Producer/Consumer clients.
Regarding using something like maven-exec, I'm just not a fan. What port do you run the broker on? How do the files get cleaned up after the run? How do you run the tests from within the IDE? What if you want to step through the broker code? Most of these are solvable but I guess this comes down to your opinion of using unit test frameworks for what are really integration-level tests. We've found it to work very well, the only real issue is that you generally have to use the same broker and client version due to the former's dependency on the latter, even if you don't use that broker version in production. I also don't quite understand the idea that the API is insufficient. How is it less sufficient than the API you get running a "real" broker in a separate process? -Tommy On Fri, 2019-12-06 at 12:59 -0600, John Roesler wrote: [EXTERNAL EMAIL] Attention: This email was sent from outside TiVo. DO NOT CLICK any links or attachments unless you expected them. ________________________________ Hi Tommy, There are some practically ancient discussions around doing this exact thing, but they haven't generally come to fruition because EmbeddedKafkaCluster itself isn't really a slam dunk, usability-wise. You'll notice that copying it (i.e., using it _at all_ ) is the option of last resort in my prior message. Clearly, we use it (heavily) in the Apache Kafka project's tests, but I can tell you that it's kind of a source of "constant sorrow". It's not that it doesn't "work" (it does), but it's more that it doesn't provide the "right" support for higher level testing of a Streams application (as opposed to testing the Streams framework itself). This is why we provided TopologyTestDriver. To provide you with the ability to test your application code in an efficient and sane manner. As it stands, supposing you really can't use TopologyTestDriver, EmbeddedKafkaCluster offers no practical benefits over just having a broker running locally for your integration tests. My thinking is, why not just do that? You can use the maven-exec-plugin, for example, to start and stop the broker automatically for your tests. By the way, I'm not saying that we should not offer a lower-level testing support utlity, it's just that I don't think we should just move EmbeddedKafkaCluster into the public API. Particularly, for testing, we need something more efficient and that can be more synchronous, but still presents the same API as a broker. This would be a ton of work to design an build, though, which I assume is why no one has done it. Thanks, -John On Fri, Dec 6, 2019, at 12:21, Thomas Becker wrote: > > Personally, I would love to see EmbeddedKafkaCluster moved to a public > test artifact, similarly to kafka-streams-test-utils. Having to > copy/paste internal classes into your project is...unfortunate. > > > On Fri, 2019-12-06 at 10:42 -0600, John Roesler wrote: > > [EXTERNAL EMAIL] Attention: This email was sent from outside TiVo. DO NOT > > CLICK any links or attachments unless you expected them. > > ________________________________ > > > > Hi Matthias! > > Thanks for the note, and the kind sentiment. > > We're always open to improvements like this, so your contribution would > > certainly be welcome. > > Just FYI, "public interface" changes need to go through the KIP process > > (see > > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FKAFKA%2FKafka%2BImprovement%2BProposals&data=02%7C01%7CThomas.Becker%40tivo.com%7C31e9dd9b7a6d4cb34cec08d77a7e7f84%7Cd05b7c6912014c0db45d7f1dcc227e4d%7C1%7C0%7C637112556050163031&sdata=ritsQphIw3P664vucl6rDupkIXh2pK%2FBUdJNo8cOVHo%3D&reserved=0 > > ). You could of course, open an exploratory PR and ask here for comments > > before deciding if you want to make a KIP, if you prefer. Personally, I > > often find it clarifying to put together a PR concurrently with the KIP, > > because it helps to flush out any "devils in the details", and also can > > help the KIP discussion. > > Just wanted to make you aware of the process. I'm happy to help you > > navigate this process, by the way. > > Regarding the specific proposal, the number one question in my mind is > > compatibility... I.e., do we add new dependencies, or change dependencies, > > that may conflict with what's already on users' classpaths? Would the > > change result in any source-code or binary incompatibility with previous > > versions? Etc... you get the idea. > > We can make such changes, but it's a _lot_ easier to support doing it in a > > major release. Right now, that would be 3.0, which is not currently > > planned. We're currently working toward 2.5, and then 2.6, and so on, > > essentially waiting for a good reason to bump up to 3.0. > > All that said, EmbeddedKafkaCluster is an interesting case, because it's > > not actually a public API! When Bill wrote the book, he included it (I > > assume) because there was no other practical approach to testing available. > > However, since the publication, we have added an official integration > > testing framework called TopologyTestDriver. When people ask me for testing > > advice, I tell them: 1. Use TopologyTestDriver (for Streams testing) 2. If > > you need a "real" broker for your test, then set up a pre/post integration > > test hook to run Kafka independently (e.g., with Maven). 3. If that's not > > practical, then _copy_ EmbeddedKafkaCluster into your own project, don't > > depend on an internal test artifact. > > To me, this means that we essentially have free reign to make changes to > > EmbeddedKafkaCluster, since it _should_ only be used for internal testing. > > In that case, I would just evaluate the merits up bumping all the tests in > > Apache Kafka up to JUnit 5. Although that's not a public API, it might be a > > big enough change to the process to justify a design document and > > project-wide discussion. > > Well, I guess it turns out I had more to say than I initially thought... > > Sorry for rambling a bit. > > What are your thoughts? -John > > On Fri, Dec 6, 2019, at 07:05, Matthias Merdes wrote: > Hi all, > > when > > reading ‘Kafka Streams in Action’ I especially enjoyed the > thoughtful > > treatment of > mocking, unit, and integration tests. > Integration testing > > in the book (and in the Kafka codebase) is done > using the > > @ClassRule-annotated EmbeddedKafkaCluster. > JUnit 5 Jupiter replaced Rules > > with a different extension model. > So my proposal would be to support a > > JUnit 5 Extension in addition to > the JUnit 4 Rule for > > EmbeddedKafkaCluster > to enable ’native’ integration in JUnit 5-based > > projects. > Being a committer with the JUnit 5 team I would be happy to > > contribute > a PR for such an extension. > Please let me know if you are > > interested. > Cheers, > Matthias > > > > > > > > > > > Matthias Merdes > > > Senior Software Architect > > > > heidelpay GmbH > Vangerowstraße 18 > > > 69115 Heidelberg > > > ------------------------------------------------------------- > +49 6221 > > 6471-692 > > > matthias.mer...@heidelpay.com<mailto:matthias.mer...@heidelpay.com> > > > -------------------------------------------------------------- > > > > Geschäftsführer: Dipl. Vw. Mirko Hüllemann, > Tamara Huber, André Munk, > > Georg Schardt > Registergericht: AG Mannheim, HRB 337681 > > -- > *Tommy Becker* > *Principal Engineer * > > *Personalized Content Discovery* > > *O* +1 919.460.4747 > *tivo.com* <http://www.tivo.com/> > > > > This email and any attachments may contain confidential and privileged > material for the sole use of the intended recipient. Any review, > copying, or distribution of this email (or any attachments) by others > is prohibited. If you are not the intended recipient, please contact > the sender immediately and permanently delete this email and any > attachments. No employee or agent of TiVo is authorized to conclude any > binding agreement on behalf of TiVo by email. Binding agreements with > TiVo may only be made by a signed written agreement. -- [cid:abe1d616b22f91e74927bd753f993a110e3dac31.camel@tivo.com] Tommy Becker Principal Engineer Personalized Content Discovery O +1 919.460.4747 tivo.com<http://www.tivo.com/> ________________________________ This email and any attachments may contain confidential and privileged material for the sole use of the intended recipient. Any review, copying, or distribution of this email (or any attachments) by others is prohibited. If you are not the intended recipient, please contact the sender immediately and permanently delete this email and any attachments. No employee or agent of TiVo is authorized to conclude any binding agreement on behalf of TiVo by email. Binding agreements with TiVo may only be made by a signed written agreement.
