Now that we are using Gradle, we can do what Robert suggests. With Maven it created a (false) cyclic dependency. It could still be a little bit confusing, but not too much.
The only modules that should currently use NeedsRunner are those that would create such a cycle. These are sdks/java/core and runners/core-construction and I see some references in sdks/java/io/google-cloud-platform and sdks/java/io/xml/. Filed https://issues.apache.org/jira/browse/BEAM-6343 and opened https://github.com/apache/beam/pull/7392. I suggest that getting the signal restored (via @Ignore) is most important, then we can un-ignore as we fix. Kenn On Fri, Dec 28, 2018 at 11:23 AM Robert Bradshaw <[email protected]> wrote: > I don't know much about these specific tests, but could we simplify > things and get rid of the whole "Needs Runner" designation by just > making the direct runner a dependency for the tests of each module > (including core)? > > On Fri, Dec 28, 2018 at 6:20 PM Gleb Kanterov <[email protected]> wrote: > > > > I looked into failing tests, we can put them into 5 categories: > > - CastTest, invalid test case, fixed in apache/beam#7372 > > - UsesMetricsPusher tests running with direct-runner, attempted to fix > in apache/beam#7374 > > - hanging tests using test stream: > BoundedReadFromUnboundedSourceTest#testTimeBound and > SplittableDoFnTest#testLateData, needs more investigation > > - test using Watch transform: WatchTest, FileIOTest, AvroIOTest. Watch > transform needs refactoring after apache/beam#6467 > > - TFRecordIOTest, needs more investigation > > > > Not sure what would be the best way to proceed while tests are failing. > > > > On Fri, Dec 28, 2018 at 5:30 PM Reuven Lax <[email protected]> wrote: > >> > >> I think they run in postcommit. Seems like this was an oversight maybe. > >> > >> On Fri, Dec 28, 2018, 5:21 PM Ismaël Mejía <[email protected] wrote: > >>> > >>> Big +1 > >>> > >>> I am in awe that they are not active anymore, they used to be in the > >>> good old maven days, when IDE integration worked and most JIRAs were > >>> not about build or test issues. Sigh. > >>> > >>> On Fri, Dec 28, 2018 at 3:24 PM Reuven Lax <[email protected]> wrote: > >>> > > >>> > Kenn and I both noticed that some needsRunner tests time out, and we > were both wondering why our PreCommit was still green. This tests are meant > to be quick, and IMO should definitely be part of Java PreCommit. > >>> > > >>> > On Fri, Dec 28, 2018 at 2:59 PM Gleb Kanterov <[email protected]> > wrote: > >>> >> > >>> >> After reading Beam Testing I had an impression that NeedsRunner > tests are executed as a part of Java PreCommit using Direct runner. > However, it doesn't seem to be the case. I've tried running these tests > locally, and few of them are failing or timeout. > >>> >> > >>> >> My proposal is adding ":beam-runners-direct-java:needsRunnerTests" > to "Run Java PreCommit" Jenkins job, or running them in parallel as a part > of a new task, not to increase build time. Running needsRunnerTests on my > laptop took 3 minutes, so I just lean towards adding them to javaPreCommit. > Thoughts? > >>> >> > >>> >> > Task :beam-runners-direct-java:needsRunnerTests > >>> >> > >>> >> org.apache.beam.runners.core.metrics.MetricsPusherTest > test FAILED > >>> >> java.lang.AssertionError at MetricsPusherTest.java:71 > >>> >> > >>> >> org.apache.beam.sdk.io.TFRecordIOTest > testReadInvalidDataMask > FAILED > >>> >> java.lang.AssertionError > >>> >> > >>> >> org.apache.beam.sdk.io.TFRecordIOTest > testReadInvalidLengthMask > FAILED > >>> >> java.lang.AssertionError > >>> >> > >>> >> org.apache.beam.sdk.io.AvroIOTest$NeedsRunnerTests > > testContinuouslyWriteAndReadMultipleFilepatterns[0: true] FAILED > >>> >> java.lang.IllegalArgumentException at AvroIOTest.java:557 > >>> >> > >>> >> org.apache.beam.sdk.io.AvroIOTest$NeedsRunnerTests > > testWriteThenReadJavaClass[0: true] FAILED > >>> >> java.lang.AssertionError at AvroIOTest.java:306 > >>> >> > >>> >> org.apache.beam.sdk.io.AvroIOTest$NeedsRunnerTests > > testContinuouslyWriteAndReadMultipleFilepatterns[1: false] FAILED > >>> >> java.lang.IllegalArgumentException at AvroIOTest.java:557 > >>> >> > >>> >> org.apache.beam.sdk.io.AvroIOTest$NeedsRunnerTests > > testWriteThenReadJavaClass[1: false] FAILED > >>> >> java.lang.AssertionError at AvroIOTest.java:306 > >>> >> > >>> >> org.apache.beam.sdk.io.FileIOTest > testMatchWatchForNewFiles > FAILED > >>> >> java.lang.IllegalArgumentException at FileIOTest.java:192 > >>> >> > >>> >> org.apache.beam.sdk.io.TextIOReadTest$BasicIOTest > > testReadWatchForNewFiles FAILED > >>> >> java.lang.IllegalArgumentException at TextIOReadTest.java:846 > >>> >> > >>> >> org.apache.beam.sdk.schemas.transforms.CastTest > > testTypeNarrowFail FAILED > >>> >> java.lang.AssertionError > >>> >> > >>> >> org.apache.beam.sdk.schemas.transforms.CastTest > > testWeakedNullableFail FAILED > >>> >> java.lang.AssertionError > >>> >> > >>> >> org.apache.beam.sdk.transforms.SplittableDoFnTest > testBoundedness > FAILED > >>> >> > org.apache.beam.sdk.testing.TestPipeline$PipelineRunMissingException > >>> >> > >>> >> org.apache.beam.sdk.transforms.SplittableDoFnTest > testLateData > FAILED > >>> >> org.junit.runners.model.TestTimedOutException at > SplittableDoFnTest.java:683 > >>> >> > >>> >> org.apache.beam.sdk.transforms.WatchTest > > testSinglePollMultipleInputsWithSideInput FAILED > >>> >> java.lang.IllegalArgumentException at WatchTest.java:112 > >>> >> > >>> >> org.apache.beam.sdk.transforms.WatchTest > > testMultiplePollsWithKeyExtractor FAILED > >>> >> java.lang.IllegalArgumentException at WatchTest.java:203 > >>> >> > >>> >> org.apache.beam.sdk.transforms.WatchTest > > testSinglePollMultipleInputs FAILED > >>> >> java.lang.IllegalArgumentException at WatchTest.java:82 > >>> >> > >>> >> org.apache.beam.sdk.transforms.WatchTest > > testMultiplePollsWithTerminationDueToTerminationCondition FAILED > >>> >> java.lang.IllegalArgumentException at WatchTest.java:150 > >>> >> > >>> >> org.apache.beam.sdk.transforms.WatchTest > > testMultiplePollsWithManyResults FAILED > >>> >> java.lang.IllegalArgumentException at WatchTest.java:308 > >>> >> > >>> >> org.apache.beam.sdk.transforms.WatchTest > > testSinglePollWithManyResults FAILED > >>> >> java.lang.IllegalArgumentException at WatchTest.java:258 > >>> >> > >>> >> org.apache.beam.sdk.transforms.WatchTest > > testMultiplePollsStopAfterTimeSinceNewOutput FAILED > >>> >> java.lang.IllegalArgumentException at WatchTest.java:232 > >>> >> > >>> >> org.apache.beam.sdk.transforms.WatchTest > > testMultiplePollsWithTerminationBecauseOutputIsFinal FAILED > >>> >> java.lang.IllegalArgumentException at WatchTest.java:150 > >>> >> > >>> >> 647 tests completed, 21 failed, 4 skipped > > > > > > > > -- > > Cheers, > > Gleb >
