[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139642#comment-16139642
 ] 

ASF GitHub Bot commented on BEAM-2671:
--

Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/3738


> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Stas Levin
>  Labels: watermark
> Fix For: 2.2.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134569#comment-16134569
 ] 

ASF GitHub Bot commented on BEAM-2671:
--

GitHub user staslev opened a pull request:

https://github.com/apache/beam/pull/3738

[BEAM-2671] Fixing CreateStreamTest#testFirstElementLate

Follow this checklist to help us incorporate your contribution quickly and 
easily:

 - [ ] Make sure there is a [JIRA 
issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the 
change (usually before you start working on it).  Trivial changes like typos do 
not require a JIRA issue.  Your pull request should address just this issue, 
without pulling in other changes.
 - [ ] Each commit in the pull request should have a meaningful subject 
line and body.
 - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue.
 - [ ] Write a pull request description that is detailed enough to 
understand what the pull request does, how, and why.
 - [ ] Run `mvn clean verify` to make sure basic checks pass. A more 
thorough check will be performed on your pull request automatically.
 - [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/staslev/beam 
BEAM-2671-fixing-CreateStreamTest#testFirstElementLate

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3738.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3738


commit 940e7a159428f18c40a001e34ebcf3e847329f3f
Author: Stas Levin 
Date:   2017-08-20T13:48:57Z

Implemented an InputDStream that syncs up with the watermark values, this 
should help with streaming tests in spark-runner.

commit 1571ea668dac45d3db6a59a5bdf2e3af43120870
Author: Stas Levin 
Date:   2017-08-20T18:54:52Z

Fixed failing SparkPipelineStateTest tests.




> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.2.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[Stas Levin (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129014#comment-16129014
 ] 

Stas Levin commented on BEAM-2671:
--

I believe I've been able to pinpoint the problem and am working towards a fix.
In case my hypothesis is correct hopefully I'll be able to resolve this within 
a week or so time - I'll keep this thread updated with the findings.

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.2.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[Reuven Lax (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129000#comment-16129000
 ] 

Reuven Lax commented on BEAM-2671:
--

Given how long 2.1.0 has taken to release, I suspect we'll cut 2.2.0 
immediately after 2.1.0 is approved. Given that, is it reasonable to fix this 
for 2.2.0?

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.2.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[Kenneth Knowles (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127927#comment-16127927
 ] 

Kenneth Knowles commented on BEAM-2671:
---

[~staslev] previously, timers were each processed separately. The state for a 
window was cleared when a timer arrived with timestamp == GC time.  This was 
actually wrong relative to the spec, and led to the wrong number of outputs in 
some situations where the end-of-window and GC timer came in the same bundle. 
You'd get multiple outputs (which is OK but wasteful) but they would be labeled 
wrong about whether they are the first or final output.

Now, timers for a window's lifecycle are processed all at once. In fact, the 
timers themselves are irrelevant. When they arrive, the window is "activated" 
and the state for the window is cleared if the watermark is far enough along 
that the window is expired. All the PaneInfo is automatically fixed for the 
corner cases about when the EOW and GC timers come together or very delayed, 
etc.

Technically, the GC timer should only ever be delivered when the watermark is 
that far along, so the actual GC time is the same.

If a runner was delivering the GC timer early, then it would have worked in the 
old logic, but won't GC in the new logic. If a timer comes in with timestamp == 
GC time but the watermark is actually not far enough along to safely GC, it 
will not cause a GC. It would be a bug to deliver that timer, and could cause 
other erroneous results due to early clearing of state - data would come in, 
would not be dropped, and would then be output. But mostly it would be rare to 
see the failure. Now you'd see it all the time.

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.2.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[Stas Levin (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124897#comment-16124897
 ] 

Stas Levin commented on BEAM-2671:
--

[~kenn], could you elaborate on when there might be a race, and what triggers 
will be involved in such a race?

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.2.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[Aviem Zur (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118451#comment-16118451
 ] 

Aviem Zur commented on BEAM-2671:
-

[~jkff] we've been running Spark runner in streaming mode using 2.0.0 
We've encountered the issue with the watermarks not advancing which was a no-go 
for us, since results were not emitted and eventually apps failed on out of 
memory exceptions. This was the reason for the fix in 
https://github.com/apache/beam/pull/3343

After integrating this fix (on our local fork) we were able to run applications 
successfully.
Regarding running the current state of master, we are not, so hard to comment 
on if the failing tests cause real applications to fail, this will have to be 
assessed. What we do know is the tests are failing.

Regarding ValidatesRunner tests, while these do pass, IIRC the creation of the 
streaming tests for Spark runner were in fact due to insufficient tests for 
Spark runner's streaming mode. So while they are not perfect and flaky, they do 
provide value. Perhaps [~amitsela] can elaborate on these tests.

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.2.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[Etienne Chauchot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118355#comment-16118355
 ] 

Etienne Chauchot commented on BEAM-2671:


I think Nexmark (https://github.com/apache/beam/pull/3114) can be of great help 
here. It has a very comprehensive test suite in both batch and streaming modes. 
I'll run the Nexmark test suite for Spark streaming against v2.1.0-RC2 and keep 
you posted.

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.2.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[JIRA]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117853#comment-16117853
 ] 

Jean-Baptiste Onofré commented on BEAM-2671:


I bump this to 2.2.0 and I will cut RC3 as it is. Yes, I ran streaming 
pipelines with 2.1.0-SNAPSHOT: it's not very complex but it worked (reading 
from JMS, doing a ParDo(DoFn) and writing the result to sharded TextIO).

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.2.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[Kenneth Knowles (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117523#comment-16117523
 ] 

Kenneth Knowles commented on BEAM-2671:
---

It is entirely possible that 935c077341de5804b29ffee3926795acf403 could 
expose existing bugs in timer handling. In particular, it changes window GC 
from being based on the target time of the timer to being based on the 
watermark. So it might turn a race condition causing flakiness into a 
guaranteed early GC. (I haven't looked more closely yet)

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.1.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[Eugene Kirpichov (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117469#comment-16117469
 ] 

Eugene Kirpichov commented on BEAM-2671:


Aviem - my apologies for misunderstanding previous comments on the JIRA. Could 
you elaborate on "If we want streaming to be working in Spark runner in 2.1.0 
we should fix this for 2.1.0" - I'd like to understand to what extent is Spark 
runner streaming currently broken in 2.1.0, given that it passes the other SDK 
ValidatesRunner tests. Are those tests insufficient / should we have more 
coverage there?

Has anybody run any serious Spark streaming pipelines on a cluster against 
2.1.0 RC2? JB - I believe you said you did; can you detail what kind of 
pipeline you ran, and does it potentially address Aviem's concerns?

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.1.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[JIRA]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117127#comment-16117127
 ] 

Jean-Baptiste Onofré commented on BEAM-2671:


Actually, as the test is flacky, it makes {{git bisect}} not accurate. I do a 
new one with several run to be sure.

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.1.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[JIRA]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16116860#comment-16116860
 ] 

Jean-Baptiste Onofré commented on BEAM-2671:


I did a new `git bisect` in the meaning of finding the first commit which 
change from Spark runner validate from success to failure. And here's the first 
one I found:

{code}
1c1f239501349f5120b0d619c4eea9c435500b78 is the first bad commit
commit 1c1f239501349f5120b0d619c4eea9c435500b78
Author: Kenneth Knowles 
Date:   Thu Jun 22 12:52:42 2017 -0700

ReduceFnTester can advance clocks without firing timers

:04 04 888e75a5513b7171cb924d3c9b883a42ee59f045 
6b6ea07557d0d57a9849afec02e1555e3f1d1334 M  runners
{code}

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.1.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[JIRA]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16116469#comment-16116469
 ] 

Jean-Baptiste Onofré commented on BEAM-2671:


FYI, [~staslev] and I are working on the issue.

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.1.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[Aviem Zur (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115339#comment-16115339
 ] 

Aviem Zur commented on BEAM-2671:
-

If we want streaming to be working in Spark runner in 2.1.0 we should fix this 
for 2.1.0
This was the goal of https://github.com/apache/beam/pull/3343
There has been regression since.

If there is a different, more important goal for this version, then we could 
release yet another version with streaming not working in spark runner.

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.1.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[JIRA]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115338#comment-16115338
 ] 

Jean-Baptiste Onofré commented on BEAM-2671:


[~aviemzur] do we target to fix that for 2.1.0 or do we bump to 2.2.0 ? Up to 
you !

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.1.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[Aviem Zur (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115336#comment-16115336
 ] 

Aviem Zur commented on BEAM-2671:
-

[~jkff] Please see the comments made by JB and myself which help demonstrate 
that that commit is not the issue.
I understand it fails on your local machine, that suite of tests is a flaky one.
Please see the link to the last successful build on Jenkins, which includes 
this commit.

JB has shared the commit at which master is failing with the current failure.
It also fails before that commit so it is hard to say that that specific commit 
introduced the issue, or rather a previous commit.

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.1.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[Eugene Kirpichov (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115164#comment-16115164
 ] 

Eugene Kirpichov commented on BEAM-2671:


20820fa5477ffcdd4a9ef2e9340353ed3c5691a9, mentioned above, is the culprit.

At 20820fa5, I ran the test 10 times and it failed 10/10.
At the previous commit b3099bba, it succeeds 10/10.

However, I did git cherry-pick --reverse of that commit on top of current 
master branch, and the test still fails - so reverting the commit is not 
sufficient.

Moreover, according to my understanding of its enclosing PR 
https://github.com/apache/beam/pull/3343, reverting it is really not an option, 
because before that PR the Spark runner was completely unusable in streaming 
mode (not advancing input watermark). Unfortunately the PR does not seem to 
include any tests for the fix, but I presume it was tested manually, or perhaps 
gets some testing coverage from ValidatesRunner tests.

I propose to postpone the current JIRA to 2.2.0, because:
- Spark Runner currently passes all ValidatesRunner tests (this is not a 
ValidatesRunner test), so I presume that it was completely broken, we'd have 
other evidence.
- Release 2.0.0 does not contain https://github.com/apache/beam/pull/3343 which 
means that in that release Spark runner *is* unusable in streaming mode, i.e. 
even if the current JIRA signals that it's still broken in some way, it can't 
be more broken than 2.0.0 - i.e. there is no regression, and there is most 
likely some improvement (albeit incomplete).



> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.1.0, 2.2.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[JIRA]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114481#comment-16114481
 ] 

Jean-Baptiste Onofré commented on BEAM-2671:


After a Apache Karaf release, I'm back on this issue. Any help is welcome.

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.1.0, 2.2.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[JIRA]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110962#comment-16110962
 ] 

Jean-Baptiste Onofré commented on BEAM-2671:


Just to be clear. I checked out on commit 
935c077341de5804b29ffee3926795acf403 and ran a build with validate runner. 
Result is:

{code}
[INFO] Running 
org.apache.beam.runners.spark.translation.streaming.CreateStreamTest
[ERROR] Tests run: 10, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 30.695 
s <<< FAILURE! - in 
org.apache.beam.runners.spark.translation.streaming.CreateStreamTest
[ERROR] 
testFirstElementLate(org.apache.beam.runners.spark.translation.streaming.CreateStreamTest)
  Time elapsed: 3.198 s  <<< FAILURE!
java.lang.AssertionError: 
Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
Expected: iterable over [] in any order
 but: Not matched: "late"
at 
org.apache.beam.sdk.testing.PAssert$PAssertionSite.capture(PAssert.java:170)
at org.apache.beam.sdk.testing.PAssert.that(PAssert.java:366)
at org.apache.beam.sdk.testing.PAssert.that(PAssert.java:358)
at 
org.apache.beam.runners.spark.translation.streaming.CreateStreamTest.testFirstElementLate(CreateStreamTest.java:233)
...
{code}

corresponding to the same failure we have on master.

Now, if I checkout on the commit just before 
935c077341de5804b29ffee3926795acf403 
(bd631b89a8434f0756e1596875e89013fb623ab5) and I do the same build, I have:

{code}
[INFO] Running 
org.apache.beam.runners.spark.translation.streaming.CreateStreamTest
[ERROR] Tests run: 10, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 55.192 
s <<< FAILURE! - in 
org.apache.beam.runners.spark.translation.streaming.CreateStreamTest
[ERROR] 
testMultipleStreams(org.apache.beam.runners.spark.translation.streaming.CreateStreamTest)
  Time elapsed: 3.689 s  <<< FAILURE!
java.lang.AssertionError:
WindowStrings/Flatten.PCollections.out:
Expected: iterable over ["foo", "bar"] in any order
 but: No item matches: "foo", "bar" in []
at 
org.apache.beam.sdk.testing.PAssert$PAssertionSite.capture(PAssert.java:170)
at org.apache.beam.sdk.testing.PAssert.that(PAssert.java:366)
at org.apache.beam.sdk.testing.PAssert.that(PAssert.java:358)
at 
org.apache.beam.runners.spark.translation.streaming.CreateStreamTest.testMultipleStreams(CreateStreamTest.java:284)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
...
{code}

So, {{CreateStreamTest}} also fails but with a different cause and on 
{{testMultipleStream}} (whereas on master and on commit mentioned above, it 
fails on {{testFirstElementLate}}).



> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.1.0, 2.2.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[JIRA]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110272#comment-16110272
 ] 

Jean-Baptiste Onofré commented on BEAM-2671:


I found the issue which cause the issue:

{code}
commit 935c077341de5804b29ffee3926795acf403
Author: Kenneth Knowles 
Date:   Thu Jun 22 18:43:39 2017 -0700

Process timer firings for a window together
{code}

The {{CreateStreamTest}} worked just before this commit and started to fail 
after this commit. I'm digging around that.

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.1.0, 2.2.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[JIRA]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106455#comment-16106455
 ] 

Jean-Baptiste Onofré commented on BEAM-2671:


I tested locally with a git bisect. Let me try another bisect after this commit.

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.1.0, 2.2.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[Aviem Zur (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106440#comment-16106440
 ] 

Aviem Zur commented on BEAM-2671:
-

20820fa5477ffcdd4a9ef2e9340353ed3c5691a9
added an element of flakiness to our build which is why we decided to move it 
to the Spark runner postcommit tests in 22dbb500289675fe95b6d149c8550e09dc26feac

However, it appears that now the Spark runner tests are failing consistently, 
which IIRC correctly was not the case.
The build history doesn't go far back enough to show when the tests started 
failing but it does keep the last successful build
which built the revision 3785b5baf4d3c47ea6625fff9babe5854e2e5fdb which 
happened after the commits mentioned above 
https://builds.apache.org/view/Beam/job/beam_PostCommit_Java_ValidatesRunner_Spark/2446/

There are 2 possibilities:
1. Something in Jenkins env has changed in a way that the flakiness occurs 100% 
of the time now.
2. A commit has been made since then that makes the tests fail consistently.

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.1.0, 2.2.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[JIRA]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106228#comment-16106228
 ] 

Jean-Baptiste Onofré commented on BEAM-2671:


Test is OK on this commit: 20820fa5477ffcdd4a9ef2e9340353ed3c5691a9.

I'm testing the next one.

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.1.0, 2.2.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[Eugene Kirpichov (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106185#comment-16106185
 ] 

Eugene Kirpichov commented on BEAM-2671:


It's overwhelmingly likely the last one as I mentioned above.

On Sat, Jul 29, 2017, 9:50 AM Jean-Baptiste Onofré (JIRA) 



> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.1.0, 2.2.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[JIRA]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106176#comment-16106176
 ] 

Jean-Baptiste Onofré commented on BEAM-2671:


It seems that 3785b5baf4d3c47ea6625fff9babe5854e2e5fdb is the first bad commit.

I'm suspecting one of the following (especially the ones specific to the Spark 
runner):

{code}
commit 3785b5baf4d3c47ea6625fff9babe5854e2e5fdb
Merge: e2ee595 4c48815
Author: Ahmet Altay 
Date:   Thu Jun 22 10:47:00 2017 -0700

This closes #3421

commit 4c488152c45ac7a6c344a21b67c968b97bf5066c
Author: Sourabh Bajaj 
Date:   Thu Jun 22 08:33:31 2017 -0700

[BEAM-1585] Fix the beam plugins installation

commit e2ee59557befd729e24e62a4991a76bab64f5755
Merge: b3099bb 22dbb50
Author: Aviem Zur 
Date:   Thu Jun 22 15:33:26 2017 +0300

This closes #3343

commit 22dbb500289675fe95b6d149c8550e09dc26feac
Author: Aviem Zur 
Date:   Wed Jun 21 17:53:21 2017 +0300

Move Spark runner streaming tests to post commit.

commit 20820fa5477ffcdd4a9ef2e9340353ed3c5691a9
Author: Aviem Zur 
Date:   Mon Jun 12 17:04:00 2017 +0300

[BEAM-2359] Fix watermark broadcasting to executors in Spark runner
{code}

I'm testing those commits.

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.1.0, 2.2.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[Eugene Kirpichov (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105918#comment-16105918
 ] 

Eugene Kirpichov commented on BEAM-2671:


Any updates?

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.1.0, 2.2.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[JIRA]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105136#comment-16105136
 ] 

Jean-Baptiste Onofré commented on BEAM-2671:


I'm doing a git bisect to identify where it has been broken. Update coming soon.

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.1.0, 2.2.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[Eugene Kirpichov (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105124#comment-16105124
 ] 

Eugene Kirpichov commented on BEAM-2671:


I'm not sure about bumping it to 2.2.0. It looks quite dangerous and I'd be 
hesitant to ignore it without checking how dangerous it actually is. Do we have 
other tests that verify that Spark runner does not incorrectly drop data in 
streaming pipelines?

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.1.0, 2.2.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[JIRA]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16104590#comment-16104590
 ] 

Jean-Baptiste Onofré commented on BEAM-2671:


I don't think this one is blocker for RC3. It would be great to have it fixed 
but not blocker. I'm waiting couple of hours and I will bump the issue for 
2.2.0.

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.1.0, 2.2.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[Eugene Kirpichov (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16104459#comment-16104459
 ] 

Eugene Kirpichov commented on BEAM-2671:


I wonder if these tests have something to do with 
https://github.com/apache/beam/commit/20820fa5477ffcdd4a9ef2e9340353ed3c5691a9 
(part of PR https://github.com/apache/beam/pull/3343 ) - it seems the last 
stable build of Spark ValidatesRunner was right before this commit 
https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastStableBuild/

[~aviemzur] , could you take a look perhaps?

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.1.0, 2.2.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[Eugene Kirpichov (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103911#comment-16103911
 ] 

Eugene Kirpichov commented on BEAM-2671:


Actually that test is https://issues.apache.org/jira/browse/BEAM-1868 - does 
that mean it also should be fixed for 2.1.0 RC3?

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.1.0, 2.2.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[Eugene Kirpichov (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103907#comment-16103907
 ] 

Eugene Kirpichov commented on BEAM-2671:


It's interesting that testMultiOutputParDo succeeds if you comment out any 
_one_ of the PAssert's - on the main tag or the additional tag, but fails if 
you keep both.

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.1.0, 2.2.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[Eugene Kirpichov (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103868#comment-16103868
 ] 

Eugene Kirpichov commented on BEAM-2671:


CreateStreamTest.testMultiOutputParDo() is failing as well.

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
> Fix For: 2.1.0, 2.2.0
>
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[JIRA]

[ 
https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16098110#comment-16098110
 ] 

Jean-Baptiste Onofré commented on BEAM-2671:


We saw that and IMHO, it should be fix for 2.1.0 RC3. I gonna take a look.

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark 
> runner
> 
>
> Key: BEAM-2671
> URL: https://issues.apache.org/jira/browse/BEAM-2671
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Etienne Chauchot
>Assignee: Jean-Baptiste Onofré
>
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>  but: Not matched: "late"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)