Thanks Stephan for pointing this out. I agree with you. +1 @Chiwan: Good idea with the Wiki. Actually maybe even better to add it to the contribution guide? Do you have time to open a PR with Stephan’s suggestions?
@Martin: I agree that it does not suffice to just point this out as a new guideline. I think the main problem is that it is very time consuming and error prone. We have seen some “minor refactorings” lately, which looked harmless, but actually introduced bugs. This is a danger as well with refactoring tests (we refactor them, but don’t have the same amount of test coverage, which results in bugs at some point in time in the future). Are there any known “heavy hitters”, which take a lot of time, but which could be tested in unit tests instead? I would start with those if you want to do it. But in general I would do this incrementally instead of aiming for a complete rewrite. – Ufuk > On 19 Sep 2015, at 10:53, Martin Liesenberg <martin.liesenb...@gmail.com> > wrote: > > Should there be a concerted effort to reduce the amount of unnecessary > integration tests and cover those cases by unit tests? > > We could collect the cases in a ticket and work through the list one by > one, no? > > Best regards, > Martin > > Chiwan Park <chiwanp...@apache.org> schrieb am Fr., 18. Sep. 2015 um > 12:33 Uhr: > >> Hi Stephan, >> >> Thanks for nice guide! I think we can upload this to the wiki or how to >> contribute documentation. >> This guide would be helpful for newcomers. >> >> Regards, >> Chiwan Park >> >>> On Sep 17, 2015, at 9:33 PM, Stephan Ewen <se...@apache.org> wrote: >>> >>> Hi all! >>> >>> The build time of Flink with all tests is nearing 1h on Travis for the >>> shortest run. >>> It is good that we do excessive testing, there are many mechanisms that >>> need that. >>> >>> I have also seen that a lot of fixes that could be tested in a UnitTest >>> style are actually tested as a full Flink program (Integration test >> style) >>> >>> While these tests are always easier to write, they have two problems: >>> - The bring up the build time by about 5 secs per test >>> - They are often not as targeted to the problem as a UnitTest >>> >>> I would like to encourage everyone to keep this in mind and do Unit tests >>> in the cases where they are the preferred choice. Please also keep that >> in >>> mind when reviewing pull requests. >>> >>> For Example: >>> - API / TypeInformation changes can be very well tested without running >>> the program. Simply create the program and test the operator's type info. >>> - Custom functions can be very well tested in isolation >>> - Input/Output formats actually test well in UnitTests. >>> >>> Integration tests need to be used when verifying behavior across >> components >>> / layers, so keep using them when they need to be used. >>> >>> >>> Greetings, >>> Stephan >> >> >> >> >> >>
