[
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16115164#comment-16115164
]
Eugene Kirpichov commented on BEAM-2671:
----------------------------------------
20820fa5477ffcdd4a9ef2e9340353ed3c5691a9, mentioned above, is the culprit.
At 20820fa5, I ran the test 10 times and it failed 10/10.
At the previous commit b3099bba, it succeeds 10/10.
However, I did git cherry-pick --reverse of that commit on top of current
master branch, and the test still fails - so reverting the commit is not
sufficient.
Moreover, according to my understanding of its enclosing PR
https://github.com/apache/beam/pull/3343, reverting it is really not an option,
because before that PR the Spark runner was completely unusable in streaming
mode (not advancing input watermark). Unfortunately the PR does not seem to
include any tests for the fix, but I presume it was tested manually, or perhaps
gets some testing coverage from ValidatesRunner tests.
I propose to postpone the current JIRA to 2.2.0, because:
- Spark Runner currently passes all ValidatesRunner tests (this is not a
ValidatesRunner test), so I presume that it was completely broken, we'd have
other evidence.
- Release 2.0.0 does not contain https://github.com/apache/beam/pull/3343 which
means that in that release Spark runner *is* unusable in streaming mode, i.e.
even if the current JIRA signals that it's still broken in some way, it can't
be more broken than 2.0.0 - i.e. there is no regression, and there is most
likely some improvement (albeit incomplete).
> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark
> runner
> --------------------------------------------------------------------------------
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
> Issue Type: Bug
> Components: runner-spark
> Reporter: Etienne Chauchot
> Assignee: Jean-Baptiste Onofré
> Fix For: 2.1.0, 2.2.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0:
> Expected: iterable over [] in any order
> but: Not matched: "late"
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
