[
https://issues.apache.org/jira/browse/BEAM-5070?focusedWorklogId=134713&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-134713
]
ASF GitHub Bot logged work on BEAM-5070:
----------------------------------------
Author: ASF GitHub Bot
Created on: 14/Aug/18 20:48
Start Date: 14/Aug/18 20:48
Worklog Time Spent: 10m
Work Description: akedin commented on issue #6224: [BEAM-5070] Disable
resumeFromCheckpoint test due to flakiness
URL: https://github.com/apache/beam/pull/6224#issuecomment-413010785
I think we should merge this PR and then fix the actual issue with the test.
But I am not sure what underlying problem the process you are describing
solves right now and how. By creating a Jira the problem is moved to another
place, potential underlying issue is not solved. And the process currently
doesn't really help to investigate and solve it . Few problems I have with it:
- I will probably forget about this issue unless it's trivial, and will
forget to not push the code to this area;
- I am not sure how much time I can / want to spend on this, and whether it
will be better to spend it on this issue vs some other task;
I think we should extend the process a bit:
- create a dashboard of disabled tests / jiras;
- define some timelines, mark disabled tests as "green / young", "red /
old";
- send the dashboard to dev@ periodically, poke assignees (needs specific
owner);
- triage the dashboard / builds for new failures (needs specific owner);
One other approach I've seen is implementing the timeouts for the disabled
tests. E.g. when disabling a test you set the deadline for the fix (specify it
with an annotation). After this deadline the test starts failing, forcing
someone to look at it.
Without extra visibility, tracking and poking people we will eventually have
a pile of jiras no-one is looking at.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 134713)
Time Spent: 1h 20m (was: 1h 10m)
> nexmark.sources.UnboundedEventSourceTest.resumeFromCheckpoint is flaky
> ----------------------------------------------------------------------
>
> Key: BEAM-5070
> URL: https://issues.apache.org/jira/browse/BEAM-5070
> Project: Beam
> Issue Type: Bug
> Components: test-failures
> Affects Versions: 2.5.0
> Reporter: Reuven Lax
> Assignee: Anton Kedin
> Priority: Critical
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> This test fails fairly frequently.
> History:
> [https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_GradleBuild/1219/testReport/junit/org.apache.beam.sdk.nexmark.sources/UnboundedEventSourceTest/resumeFromCheckpoint/history/]
> Sample job:
> https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_GradleBuild/1219/testReport/org.apache.beam.sdk.nexmark.sources/UnboundedEventSourceTest/resumeFromCheckpoint/
> Failure log:
> org.junit.ComparisonFailure:
> expected:<...":"UTC"},"afterNow":[true,"beforeNow":fals]e,"equalNow":false},...>
> but
> was:<...":"UTC"},"afterNow":[false,"beforeNow":tru]e,"equalNow":false},...>
> at org.junit.Assert.assertEquals(Assert.java:115) at
> org.junit.Assert.assertEquals(Assert.java:144) at
> org.apache.beam.sdk.nexmark.sources.UnboundedEventSourceTest$EventIdChecker.add(UnboundedEventSourceTest.java:71)
> at
> org.apache.beam.sdk.nexmark.sources.UnboundedEventSourceTest.resumeFromCheckpoint(UnboundedEventSourceTest.java:96)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
