suztomo edited a comment on issue #11090: [BEAM-9470] :sdks:java:io:kinesis:test is flaky URL: https://github.com/apache/beam/pull/11090#issuecomment-598926854 In the test case, the two iterators return the following data: - firstIterator: `throw(e)`; `[a, b]`; `[]`; `[]`; `[]`; `[]` ... - secondIterator: `[c]`; `[d]`; `[]`; `[]`; `[]` ... The test case waits for 5 elements from shardReadersPool: ``` List<KinesisRecord> fetchedRecords = new ArrayList<>(); while (fetchedRecords.size() < 4) { CustomOptional<KinesisRecord> nextRecord = shardReadersPool.nextRecord(); if (nextRecord.isPresent()) { fetchedRecords.add(nextRecord.get()); } } ``` It does not care what are the content of 5 elements. The test's expectation is to capture the following 4 elements: ``` verify(customRateLimitPolicy).onThrottle(same(e)); verify(customRateLimitPolicy).onSuccess(eq(ImmutableList.of(a, b))); verify(customRateLimitPolicy).onSuccess(eq(singletonList(c))); verify(customRateLimitPolicy).onSuccess(eq(singletonList(d))); ``` My assumption: there's no guarantee that 2 elements from firstIterator and secondIterator are served evenly when 5 elements are consumed from them. My suggestion: make the while loop until it confirms the expected 5 elements (at least [a,b] and [d]) , not just the counting of them, are in `fetchedRecords` variable. (Sorry if this is missing the point, I wrote this without checking @jfarr 's updated solution)
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
