[jira] [Commented] (FLUME-2997) Fix flaky junit test in SpillableMemoryChannel
[ https://issues.apache.org/jira/browse/FLUME-2997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15611692#comment-15611692 ] Hudson commented on FLUME-2997: --- FAILURE: Integrated in Jenkins build Flume-trunk-hbase-1 #222 (See [https://builds.apache.org/job/Flume-trunk-hbase-1/222/]) FLUME-2997. Fix flaky test in SpillableMemoryChannel (bessbd: [http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git=commit=7c4b2fd3e88f038dea3bdae01eb864547770ad48]) * (edit) flume-ng-channels/flume-spillable-memory-channel/src/test/java/org/apache/flume/channel/TestSpillableMemoryChannel.java > Fix flaky junit test in SpillableMemoryChannel > -- > > Key: FLUME-2997 > URL: https://issues.apache.org/jira/browse/FLUME-2997 > Project: Flume > Issue Type: Test >Affects Versions: v1.7.0 >Reporter: Attila Simon >Assignee: Attila Simon > Fix For: v1.8.0 > > Attachments: FLUME-2997-1.patch, FLUME-2997.patch > > > testParallelSingleSourceAndSink sometimes trigger an edge case scenario if > sinks are slower than sources. In such situations the channel can get full > thus uncaught ChannelFullException breaks the test. Since > testCapacityWithOverflow was designed to cover such edge-case scenario > already we can safely fix the test by increasing the channel capacity to make > sure it won't get full. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLUME-2997) Fix flaky junit test in SpillableMemoryChannel
[ https://issues.apache.org/jira/browse/FLUME-2997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15611633#comment-15611633 ] ASF subversion and git services commented on FLUME-2997: Commit 7c4b2fd3e88f038dea3bdae01eb864547770ad48 in flume's branch refs/heads/trunk from [~sati] [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=7c4b2fd ] FLUME-2997. Fix flaky test in SpillableMemoryChannel When the mock sinks are slower than sources, testParallelSingleSourceAndSink sometimes fails. In such situations the channel can get full, thus uncaught ChannelFullException breaks the test. Since testCapacityWithOverflow was designed to cover such a scenario, we can safely fix the test by increasing the channel capacity to make sure it won't get full. This commit does the previously mentioned change. Reviewers: Bessenyei Balázs Donát (Attila Simon via Bessenyei Balázs Donát) > Fix flaky junit test in SpillableMemoryChannel > -- > > Key: FLUME-2997 > URL: https://issues.apache.org/jira/browse/FLUME-2997 > Project: Flume > Issue Type: Test >Affects Versions: v1.7.0 >Reporter: Attila Simon >Assignee: Attila Simon > Fix For: v1.8.0 > > Attachments: FLUME-2997-1.patch, FLUME-2997.patch > > > testParallelSingleSourceAndSink sometimes trigger an edge case scenario if > sinks are slower than sources. In such situations the channel can get full > thus uncaught ChannelFullException breaks the test. Since > testCapacityWithOverflow was designed to cover such edge-case scenario > already we can safely fix the test by increasing the channel capacity to make > sure it won't get full. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLUME-2997) Fix flaky junit test in SpillableMemoryChannel
[ https://issues.apache.org/jira/browse/FLUME-2997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15611569#comment-15611569 ] Bessenyei Balázs Donát commented on FLUME-2997: --- I'm about to commit this > Fix flaky junit test in SpillableMemoryChannel > -- > > Key: FLUME-2997 > URL: https://issues.apache.org/jira/browse/FLUME-2997 > Project: Flume > Issue Type: Test >Affects Versions: v1.7.0 >Reporter: Attila Simon >Assignee: Attila Simon > Fix For: v1.8.0 > > Attachments: FLUME-2997-1.patch, FLUME-2997.patch > > > testParallelSingleSourceAndSink sometimes trigger an edge case scenario if > sinks are slower than sources. In such situations the channel can get full > thus uncaught ChannelFullException breaks the test. Since > testCapacityWithOverflow was designed to cover such edge-case scenario > already we can safely fix the test by increasing the channel capacity to make > sure it won't get full. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLUME-2997) Fix flaky junit test in SpillableMemoryChannel
[ https://issues.apache.org/jira/browse/FLUME-2997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15585137#comment-15585137 ] Attila Simon commented on FLUME-2997: - I think I missed some description of this change so let me provide it now: Originally this test case aimed to verify behaviour about having multiple mocked sources putting events on the same SpillableMemoryChannel while multiple mocked sinks drains those events. Given the mock implementation didn't do retry if sources are faster than the sinks the channel will get full. Since full channel doesn't accept new events the mocked sources fail their task by raising a runtime exception. This edge case should be avoided thus the increased capacity (which makes sure that channel cannot be full). The "channel is full" edge case is already covered by another test. Those System.out.println-s were replaced by asserts to make the function to became a test. It wasn't testing anything before. It only ran to the end without checking anything. (The flakiness was caused by a ChannelFullException). I guess the original author used those System.out.println-s for manual testing (purely looking at the numbers). This change also made that manual step automatic. > Fix flaky junit test in SpillableMemoryChannel > -- > > Key: FLUME-2997 > URL: https://issues.apache.org/jira/browse/FLUME-2997 > Project: Flume > Issue Type: Test >Affects Versions: v1.7.0 >Reporter: Attila Simon >Assignee: Attila Simon > Fix For: v1.8.0 > > Attachments: FLUME-2997-1.patch, FLUME-2997.patch > > > testParallelSingleSourceAndSink sometimes trigger an edge case scenario if > sinks are slower than sources. In such situations the channel can get full > thus uncaught ChannelFullException breaks the test. Since > testCapacityWithOverflow was designed to cover such edge-case scenario > already we can safely fix the test by increasing the channel capacity to make > sure it won't get full. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLUME-2997) Fix flaky junit test in SpillableMemoryChannel
[ https://issues.apache.org/jira/browse/FLUME-2997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15571828#comment-15571828 ] Bessenyei Balázs Donát commented on FLUME-2997: --- [~sati]: thank you for the patch! Can you please explain why you think it's better to remove the System.out.println-s we had in the test? > Fix flaky junit test in SpillableMemoryChannel > -- > > Key: FLUME-2997 > URL: https://issues.apache.org/jira/browse/FLUME-2997 > Project: Flume > Issue Type: Test >Affects Versions: v1.7.0 >Reporter: Attila Simon >Assignee: Attila Simon > Fix For: v1.8.0 > > Attachments: FLUME-2997-1.patch, FLUME-2997.patch > > > testParallelSingleSourceAndSink sometimes trigger an edge case scenario if > sinks are slower than sources. In such situations the channel can get full > thus uncaught ChannelFullException breaks the test. Since > testCapacityWithOverflow was designed to cover such edge-case scenario > already we can safely fix the test by increasing the channel capacity to make > sure it won't get full. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLUME-2997) Fix flaky junit test in SpillableMemoryChannel
[ https://issues.apache.org/jira/browse/FLUME-2997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523123#comment-15523123 ] Attila Simon commented on FLUME-2997: - Code change in the attached patch sets up a channel in the test with a capacity enough to store the produced events. > Fix flaky junit test in SpillableMemoryChannel > -- > > Key: FLUME-2997 > URL: https://issues.apache.org/jira/browse/FLUME-2997 > Project: Flume > Issue Type: Test >Affects Versions: v1.7.0 >Reporter: Attila Simon >Assignee: Attila Simon > Fix For: v1.7.0 > > Attachments: FLUME-2997.patch > > > testParallelSingleSourceAndSink sometimes trigger an edge case scenario if > sinks are slower than sources. In such situations the channel can get full > thus uncaught ChannelFullException breaks the test. Since > testCapacityWithOverflow was designed to cover such edge-case scenario > already we can safely fix the test by increasing the channel capacity to make > sure it won't get full. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
