[jira] [Work logged] (BEAM-3409) Unexpected behavior of DoFn teardown method running in unit tests

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3409?focusedWorklogId=87881&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-87881
 ]

ASF GitHub Bot logged work on BEAM-3409:


Author: ASF GitHub Bot
Created on: 05/Apr/18 04:09
Start Date: 05/Apr/18 04:09
Worklog Time Spent: 10m 
  Work Description: jbonofre closed pull request #4790: [BEAM-3409] 
waitUntilFinish() doesn't wait for the teardown execution on Direct runner - 
fixing compilation issue on flink
URL: https://github.com/apache/beam/pull/4790
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/runners/core-java/src/main/java/org/apache/beam/runners/core/DoFnRunner.java 
b/runners/core-java/src/main/java/org/apache/beam/runners/core/DoFnRunner.java
index 30648f6e582..fd36318e5af 100644
--- 
a/runners/core-java/src/main/java/org/apache/beam/runners/core/DoFnRunner.java
+++ 
b/runners/core-java/src/main/java/org/apache/beam/runners/core/DoFnRunner.java
@@ -49,4 +49,10 @@
* additional tasks, such as flushing in-memory states.
*/
   void finishBundle();
+
+  /**
+   * @since 2.5.0
+   * @return the underlying fn instance.
+   */
+  DoFn getFn();
 }
diff --git 
a/runners/core-java/src/main/java/org/apache/beam/runners/core/LateDataDroppingDoFnRunner.java
 
b/runners/core-java/src/main/java/org/apache/beam/runners/core/LateDataDroppingDoFnRunner.java
index f89aa4e839c..d101cc57b27 100644
--- 
a/runners/core-java/src/main/java/org/apache/beam/runners/core/LateDataDroppingDoFnRunner.java
+++ 
b/runners/core-java/src/main/java/org/apache/beam/runners/core/LateDataDroppingDoFnRunner.java
@@ -60,6 +60,11 @@ public LateDataDroppingDoFnRunner(
 lateDataFilter = new LateDataFilter(windowingStrategy, timerInternals);
   }
 
+  @Override
+  public DoFn, KV> getFn() {
+return doFnRunner.getFn();
+  }
+
   @Override
   public void startBundle() {
 doFnRunner.startBundle();
diff --git 
a/runners/core-java/src/main/java/org/apache/beam/runners/core/ProcessFnRunner.java
 
b/runners/core-java/src/main/java/org/apache/beam/runners/core/ProcessFnRunner.java
index e4dfd132e2d..8c360ef0bd0 100644
--- 
a/runners/core-java/src/main/java/org/apache/beam/runners/core/ProcessFnRunner.java
+++ 
b/runners/core-java/src/main/java/org/apache/beam/runners/core/ProcessFnRunner.java
@@ -25,6 +25,7 @@
 import org.apache.beam.runners.core.StateNamespaces.WindowNamespace;
 import org.apache.beam.runners.core.TimerInternals.TimerData;
 import org.apache.beam.sdk.state.TimeDomain;
+import org.apache.beam.sdk.transforms.DoFn;
 import org.apache.beam.sdk.transforms.windowing.BoundedWindow;
 import org.apache.beam.sdk.transforms.windowing.GlobalWindow;
 import org.apache.beam.sdk.util.WindowedValue;
@@ -52,6 +53,11 @@ public ProcessFnRunner(
 this.sideInputReader = sideInputReader;
   }
 
+  @Override
+  public DoFn>, OutputT> 
getFn() {
+return underlying.getFn();
+  }
+
   @Override
   public void startBundle() {
 underlying.startBundle();
diff --git 
a/runners/core-java/src/main/java/org/apache/beam/runners/core/PushbackSideInputDoFnRunner.java
 
b/runners/core-java/src/main/java/org/apache/beam/runners/core/PushbackSideInputDoFnRunner.java
index 8f21086794d..f3013eff0cb 100644
--- 
a/runners/core-java/src/main/java/org/apache/beam/runners/core/PushbackSideInputDoFnRunner.java
+++ 
b/runners/core-java/src/main/java/org/apache/beam/runners/core/PushbackSideInputDoFnRunner.java
@@ -48,4 +48,10 @@ void onTimer(String timerId, BoundedWindow window, Instant 
timestamp,
 
   /** Calls the underlying {@link DoFn.FinishBundle} method. */
   void finishBundle();
+
+  /**
+   * @since 2.5.0
+   * @return the underlying fn instance.
+   */
+  DoFn getFn();
 }
diff --git 
a/runners/core-java/src/main/java/org/apache/beam/runners/core/SimpleDoFnRunner.java
 
b/runners/core-java/src/main/java/org/apache/beam/runners/core/SimpleDoFnRunner.java
index d4c5775464b..7e60b033e54 100644
--- 
a/runners/core-java/src/main/java/org/apache/beam/runners/core/SimpleDoFnRunner.java
+++ 
b/runners/core-java/src/main/java/org/apache/beam/runners/core/SimpleDoFnRunner.java
@@ -120,6 +120,11 @@ public SimpleDoFnRunner(
 this.allowedLateness = windowingStrategy.getAllowedLateness();
   }
 
+  @Override
+  public DoFn getFn() {
+return fn;
+  }
+
   @Override
   public void startBundle() {
 // This can contain user code. Wrap it in case it throws an exception.
diff --git 
a/runners/core-java/src/main/java/org/apache/beam/runners/core/SimplePushbackSideInputDoFnRunner.java
 
b/runners/core-java/src/main/java/org/apache/beam/runners/core/SimplePushbackSideInputDoFnRunner.java
index 591a6a2f059..8b7489b8798 100644
--- 
a/runners

[jira] [Work logged] (BEAM-3409) Unexpected behavior of DoFn teardown method running in unit tests

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3409?focusedWorklogId=87880&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-87880
 ]

ASF GitHub Bot logged work on BEAM-3409:


Author: ASF GitHub Bot
Created on: 05/Apr/18 04:08
Start Date: 05/Apr/18 04:08
Worklog Time Spent: 10m 
  Work Description: jbonofre commented on issue #4790: [BEAM-3409] 
waitUntilFinish() doesn't wait for the teardown execution on Direct runner - 
fixing compilation issue on flink
URL: https://github.com/apache/beam/pull/4790#issuecomment-378814811
 
 
   Run Java Gradle PreCommit


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 87880)
Time Spent: 7h  (was: 6h 50m)

> Unexpected behavior of DoFn teardown method running in unit tests 
> --
>
> Key: BEAM-3409
> URL: https://issues.apache.org/jira/browse/BEAM-3409
> Project: Beam
>  Issue Type: Bug
>  Components: runner-direct
>Affects Versions: 2.3.0
>Reporter: Alexey Romanenko
>Assignee: Romain Manni-Bucau
>Priority: Blocker
>  Labels: test
> Fix For: 2.5.0
>
>  Time Spent: 7h
>  Remaining Estimate: 0h
>
> Writing a unit test, I found out a strange behaviour of Teardown method of 
> DoFn implementation when I run this method in unit tests using TestPipeline.
> To be more precise, it doesn’t wait until teardown() method will be finished, 
> it just exits from this method after about 1 sec (on my machine) even if it 
> should take longer (very simple example - running infinite loop inside this 
> method or put thread in sleep). In the same time, when I run the same code 
> from main() with ordinary Pipeline and direct runner, then it’s ok and it 
> works as expected - teardown() method will be performed completely despite 
> how much time it will take.
> I created two test cases to reproduce this issue - the first one to run with 
> main() and the second one to run with junit. They use the same implementation 
> of DoFn (class LongTearDownFn) and expects that teardown method will be 
> running at least for SLEEP_TIME ms. In case of running as junit test it's not 
> a case (see output log).
> - run with main()
> https://github.com/aromanenko-dev/beam-samples/blob/master/runners-tests/src/main/java/TearDown.java
> - run with junit
> https://github.com/aromanenko-dev/beam-samples/blob/master/runners-tests/src/test/java/TearDownTest.java



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3409) Unexpected behavior of DoFn teardown method running in unit tests

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3409?focusedWorklogId=87598&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-87598
 ]

ASF GitHub Bot logged work on BEAM-3409:


Author: ASF GitHub Bot
Created on: 04/Apr/18 15:45
Start Date: 04/Apr/18 15:45
Worklog Time Spent: 10m 
  Work Description: jbonofre commented on issue #4790: [BEAM-3409] 
waitUntilFinish() doesn't wait for the teardown execution on Direct runner - 
fixing compilation issue on flink
URL: https://github.com/apache/beam/pull/4790#issuecomment-378647810
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 87598)
Time Spent: 6h 50m  (was: 6h 40m)

> Unexpected behavior of DoFn teardown method running in unit tests 
> --
>
> Key: BEAM-3409
> URL: https://issues.apache.org/jira/browse/BEAM-3409
> Project: Beam
>  Issue Type: Bug
>  Components: runner-direct
>Affects Versions: 2.3.0
>Reporter: Alexey Romanenko
>Assignee: Romain Manni-Bucau
>Priority: Blocker
>  Labels: test
> Fix For: 2.5.0
>
>  Time Spent: 6h 50m
>  Remaining Estimate: 0h
>
> Writing a unit test, I found out a strange behaviour of Teardown method of 
> DoFn implementation when I run this method in unit tests using TestPipeline.
> To be more precise, it doesn’t wait until teardown() method will be finished, 
> it just exits from this method after about 1 sec (on my machine) even if it 
> should take longer (very simple example - running infinite loop inside this 
> method or put thread in sleep). In the same time, when I run the same code 
> from main() with ordinary Pipeline and direct runner, then it’s ok and it 
> works as expected - teardown() method will be performed completely despite 
> how much time it will take.
> I created two test cases to reproduce this issue - the first one to run with 
> main() and the second one to run with junit. They use the same implementation 
> of DoFn (class LongTearDownFn) and expects that teardown method will be 
> running at least for SLEEP_TIME ms. In case of running as junit test it's not 
> a case (see output log).
> - run with main()
> https://github.com/aromanenko-dev/beam-samples/blob/master/runners-tests/src/main/java/TearDown.java
> - run with junit
> https://github.com/aromanenko-dev/beam-samples/blob/master/runners-tests/src/test/java/TearDownTest.java



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3409) Unexpected behavior of DoFn teardown method running in unit tests

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3409?focusedWorklogId=87522&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-87522
 ]

ASF GitHub Bot logged work on BEAM-3409:


Author: ASF GitHub Bot
Created on: 04/Apr/18 12:42
Start Date: 04/Apr/18 12:42
Worklog Time Spent: 10m 
  Work Description: rmannibucau commented on issue #4790: [BEAM-3409] 
waitUntilFinish() doesn't wait for the teardown execution on Direct runner - 
fixing compilation issue on flink
URL: https://github.com/apache/beam/pull/4790#issuecomment-378586098
 
 
   fixed the @since, thanks for the catch


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 87522)
Time Spent: 6h 40m  (was: 6.5h)

> Unexpected behavior of DoFn teardown method running in unit tests 
> --
>
> Key: BEAM-3409
> URL: https://issues.apache.org/jira/browse/BEAM-3409
> Project: Beam
>  Issue Type: Bug
>  Components: runner-direct
>Affects Versions: 2.3.0
>Reporter: Alexey Romanenko
>Assignee: Romain Manni-Bucau
>Priority: Blocker
>  Labels: test
> Fix For: 2.5.0
>
>  Time Spent: 6h 40m
>  Remaining Estimate: 0h
>
> Writing a unit test, I found out a strange behaviour of Teardown method of 
> DoFn implementation when I run this method in unit tests using TestPipeline.
> To be more precise, it doesn’t wait until teardown() method will be finished, 
> it just exits from this method after about 1 sec (on my machine) even if it 
> should take longer (very simple example - running infinite loop inside this 
> method or put thread in sleep). In the same time, when I run the same code 
> from main() with ordinary Pipeline and direct runner, then it’s ok and it 
> works as expected - teardown() method will be performed completely despite 
> how much time it will take.
> I created two test cases to reproduce this issue - the first one to run with 
> main() and the second one to run with junit. They use the same implementation 
> of DoFn (class LongTearDownFn) and expects that teardown method will be 
> running at least for SLEEP_TIME ms. In case of running as junit test it's not 
> a case (see output log).
> - run with main()
> https://github.com/aromanenko-dev/beam-samples/blob/master/runners-tests/src/main/java/TearDown.java
> - run with junit
> https://github.com/aromanenko-dev/beam-samples/blob/master/runners-tests/src/test/java/TearDownTest.java



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3409) Unexpected behavior of DoFn teardown method running in unit tests

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3409?focusedWorklogId=87510&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-87510
 ]

ASF GitHub Bot logged work on BEAM-3409:


Author: ASF GitHub Bot
Created on: 04/Apr/18 11:55
Start Date: 04/Apr/18 11:55
Worklog Time Spent: 10m 
  Work Description: jbonofre commented on a change in pull request #4790: 
[BEAM-3409] waitUntilFinish() doesn't wait for the teardown execution on Direct 
runner - fixing compilation issue on flink
URL: https://github.com/apache/beam/pull/4790#discussion_r179112652
 
 

 ##
 File path: 
runners/core-java/src/main/java/org/apache/beam/runners/core/PushbackSideInputDoFnRunner.java
 ##
 @@ -48,4 +48,10 @@ void onTimer(String timerId, BoundedWindow window, Instant 
timestamp,
 
   /** Calls the underlying {@link DoFn.FinishBundle} method. */
   void finishBundle();
+
+  /**
+   * @since 2.4.0
 
 Review comment:
   Same here.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 87510)
Time Spent: 6.5h  (was: 6h 20m)

> Unexpected behavior of DoFn teardown method running in unit tests 
> --
>
> Key: BEAM-3409
> URL: https://issues.apache.org/jira/browse/BEAM-3409
> Project: Beam
>  Issue Type: Bug
>  Components: runner-direct
>Affects Versions: 2.3.0
>Reporter: Alexey Romanenko
>Assignee: Romain Manni-Bucau
>Priority: Blocker
>  Labels: test
> Fix For: 2.5.0
>
>  Time Spent: 6.5h
>  Remaining Estimate: 0h
>
> Writing a unit test, I found out a strange behaviour of Teardown method of 
> DoFn implementation when I run this method in unit tests using TestPipeline.
> To be more precise, it doesn’t wait until teardown() method will be finished, 
> it just exits from this method after about 1 sec (on my machine) even if it 
> should take longer (very simple example - running infinite loop inside this 
> method or put thread in sleep). In the same time, when I run the same code 
> from main() with ordinary Pipeline and direct runner, then it’s ok and it 
> works as expected - teardown() method will be performed completely despite 
> how much time it will take.
> I created two test cases to reproduce this issue - the first one to run with 
> main() and the second one to run with junit. They use the same implementation 
> of DoFn (class LongTearDownFn) and expects that teardown method will be 
> running at least for SLEEP_TIME ms. In case of running as junit test it's not 
> a case (see output log).
> - run with main()
> https://github.com/aromanenko-dev/beam-samples/blob/master/runners-tests/src/main/java/TearDown.java
> - run with junit
> https://github.com/aromanenko-dev/beam-samples/blob/master/runners-tests/src/test/java/TearDownTest.java



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3409) Unexpected behavior of DoFn teardown method running in unit tests

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3409?focusedWorklogId=87509&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-87509
 ]

ASF GitHub Bot logged work on BEAM-3409:


Author: ASF GitHub Bot
Created on: 04/Apr/18 11:55
Start Date: 04/Apr/18 11:55
Worklog Time Spent: 10m 
  Work Description: jbonofre commented on a change in pull request #4790: 
[BEAM-3409] waitUntilFinish() doesn't wait for the teardown execution on Direct 
runner - fixing compilation issue on flink
URL: https://github.com/apache/beam/pull/4790#discussion_r179112608
 
 

 ##
 File path: 
runners/core-java/src/main/java/org/apache/beam/runners/core/DoFnRunner.java
 ##
 @@ -49,4 +49,10 @@
* additional tasks, such as flushing in-memory states.
*/
   void finishBundle();
+
+  /**
+   * @since 2.4.0
 
 Review comment:
   It should be update to `@since 2.5.0` right ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 87509)
Time Spent: 6h 20m  (was: 6h 10m)

> Unexpected behavior of DoFn teardown method running in unit tests 
> --
>
> Key: BEAM-3409
> URL: https://issues.apache.org/jira/browse/BEAM-3409
> Project: Beam
>  Issue Type: Bug
>  Components: runner-direct
>Affects Versions: 2.3.0
>Reporter: Alexey Romanenko
>Assignee: Romain Manni-Bucau
>Priority: Blocker
>  Labels: test
> Fix For: 2.5.0
>
>  Time Spent: 6h 20m
>  Remaining Estimate: 0h
>
> Writing a unit test, I found out a strange behaviour of Teardown method of 
> DoFn implementation when I run this method in unit tests using TestPipeline.
> To be more precise, it doesn’t wait until teardown() method will be finished, 
> it just exits from this method after about 1 sec (on my machine) even if it 
> should take longer (very simple example - running infinite loop inside this 
> method or put thread in sleep). In the same time, when I run the same code 
> from main() with ordinary Pipeline and direct runner, then it’s ok and it 
> works as expected - teardown() method will be performed completely despite 
> how much time it will take.
> I created two test cases to reproduce this issue - the first one to run with 
> main() and the second one to run with junit. They use the same implementation 
> of DoFn (class LongTearDownFn) and expects that teardown method will be 
> running at least for SLEEP_TIME ms. In case of running as junit test it's not 
> a case (see output log).
> - run with main()
> https://github.com/aromanenko-dev/beam-samples/blob/master/runners-tests/src/main/java/TearDown.java
> - run with junit
> https://github.com/aromanenko-dev/beam-samples/blob/master/runners-tests/src/test/java/TearDownTest.java



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3409) Unexpected behavior of DoFn teardown method running in unit tests

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3409?focusedWorklogId=87059&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-87059
 ]

ASF GitHub Bot logged work on BEAM-3409:


Author: ASF GitHub Bot
Created on: 03/Apr/18 12:17
Start Date: 03/Apr/18 12:17
Worklog Time Spent: 10m 
  Work Description: jbonofre commented on issue #4790: [BEAM-3409] 
waitUntilFinish() doesn't wait for the teardown execution on Direct runner - 
fixing compilation issue on flink
URL: https://github.com/apache/beam/pull/4790#issuecomment-378229392
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 87059)
Time Spent: 6h 10m  (was: 6h)

> Unexpected behavior of DoFn teardown method running in unit tests 
> --
>
> Key: BEAM-3409
> URL: https://issues.apache.org/jira/browse/BEAM-3409
> Project: Beam
>  Issue Type: Bug
>  Components: runner-direct
>Affects Versions: 2.3.0
>Reporter: Alexey Romanenko
>Assignee: Romain Manni-Bucau
>Priority: Blocker
>  Labels: test
> Fix For: 2.5.0
>
>  Time Spent: 6h 10m
>  Remaining Estimate: 0h
>
> Writing a unit test, I found out a strange behaviour of Teardown method of 
> DoFn implementation when I run this method in unit tests using TestPipeline.
> To be more precise, it doesn’t wait until teardown() method will be finished, 
> it just exits from this method after about 1 sec (on my machine) even if it 
> should take longer (very simple example - running infinite loop inside this 
> method or put thread in sleep). In the same time, when I run the same code 
> from main() with ordinary Pipeline and direct runner, then it’s ok and it 
> works as expected - teardown() method will be performed completely despite 
> how much time it will take.
> I created two test cases to reproduce this issue - the first one to run with 
> main() and the second one to run with junit. They use the same implementation 
> of DoFn (class LongTearDownFn) and expects that teardown method will be 
> running at least for SLEEP_TIME ms. In case of running as junit test it's not 
> a case (see output log).
> - run with main()
> https://github.com/aromanenko-dev/beam-samples/blob/master/runners-tests/src/main/java/TearDown.java
> - run with junit
> https://github.com/aromanenko-dev/beam-samples/blob/master/runners-tests/src/test/java/TearDownTest.java



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3409) Unexpected behavior of DoFn teardown method running in unit tests

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3409?focusedWorklogId=87058&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-87058
 ]

ASF GitHub Bot logged work on BEAM-3409:


Author: ASF GitHub Bot
Created on: 03/Apr/18 12:17
Start Date: 03/Apr/18 12:17
Worklog Time Spent: 10m 
  Work Description: jbonofre commented on issue #4790: [BEAM-3409] 
waitUntilFinish() doesn't wait for the teardown execution on Direct runner - 
fixing compilation issue on flink
URL: https://github.com/apache/beam/pull/4790#issuecomment-378229368
 
 
   My bad, I looked on the old version before the rebase.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 87058)
Time Spent: 6h  (was: 5h 50m)

> Unexpected behavior of DoFn teardown method running in unit tests 
> --
>
> Key: BEAM-3409
> URL: https://issues.apache.org/jira/browse/BEAM-3409
> Project: Beam
>  Issue Type: Bug
>  Components: runner-direct
>Affects Versions: 2.3.0
>Reporter: Alexey Romanenko
>Assignee: Romain Manni-Bucau
>Priority: Blocker
>  Labels: test
> Fix For: 2.5.0
>
>  Time Spent: 6h
>  Remaining Estimate: 0h
>
> Writing a unit test, I found out a strange behaviour of Teardown method of 
> DoFn implementation when I run this method in unit tests using TestPipeline.
> To be more precise, it doesn’t wait until teardown() method will be finished, 
> it just exits from this method after about 1 sec (on my machine) even if it 
> should take longer (very simple example - running infinite loop inside this 
> method or put thread in sleep). In the same time, when I run the same code 
> from main() with ordinary Pipeline and direct runner, then it’s ok and it 
> works as expected - teardown() method will be performed completely despite 
> how much time it will take.
> I created two test cases to reproduce this issue - the first one to run with 
> main() and the second one to run with junit. They use the same implementation 
> of DoFn (class LongTearDownFn) and expects that teardown method will be 
> running at least for SLEEP_TIME ms. In case of running as junit test it's not 
> a case (see output log).
> - run with main()
> https://github.com/aromanenko-dev/beam-samples/blob/master/runners-tests/src/main/java/TearDown.java
> - run with junit
> https://github.com/aromanenko-dev/beam-samples/blob/master/runners-tests/src/test/java/TearDownTest.java



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3409) Unexpected behavior of DoFn teardown method running in unit tests

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3409?focusedWorklogId=87057&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-87057
 ]

ASF GitHub Bot logged work on BEAM-3409:


Author: ASF GitHub Bot
Created on: 03/Apr/18 12:14
Start Date: 03/Apr/18 12:14
Worklog Time Spent: 10m 
  Work Description: rmannibucau commented on issue #4790: [BEAM-3409] 
waitUntilFinish() doesn't wait for the teardown execution on Direct runner - 
fixing compilation issue on flink
URL: https://github.com/apache/beam/pull/4790#issuecomment-378228878
 
 
   @jbonofre which conflicts? rebased this morning and I dont see them on github


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 87057)
Time Spent: 5h 50m  (was: 5h 40m)

> Unexpected behavior of DoFn teardown method running in unit tests 
> --
>
> Key: BEAM-3409
> URL: https://issues.apache.org/jira/browse/BEAM-3409
> Project: Beam
>  Issue Type: Bug
>  Components: runner-direct
>Affects Versions: 2.3.0
>Reporter: Alexey Romanenko
>Assignee: Romain Manni-Bucau
>Priority: Blocker
>  Labels: test
> Fix For: 2.5.0
>
>  Time Spent: 5h 50m
>  Remaining Estimate: 0h
>
> Writing a unit test, I found out a strange behaviour of Teardown method of 
> DoFn implementation when I run this method in unit tests using TestPipeline.
> To be more precise, it doesn’t wait until teardown() method will be finished, 
> it just exits from this method after about 1 sec (on my machine) even if it 
> should take longer (very simple example - running infinite loop inside this 
> method or put thread in sleep). In the same time, when I run the same code 
> from main() with ordinary Pipeline and direct runner, then it’s ok and it 
> works as expected - teardown() method will be performed completely despite 
> how much time it will take.
> I created two test cases to reproduce this issue - the first one to run with 
> main() and the second one to run with junit. They use the same implementation 
> of DoFn (class LongTearDownFn) and expects that teardown method will be 
> running at least for SLEEP_TIME ms. In case of running as junit test it's not 
> a case (see output log).
> - run with main()
> https://github.com/aromanenko-dev/beam-samples/blob/master/runners-tests/src/main/java/TearDown.java
> - run with junit
> https://github.com/aromanenko-dev/beam-samples/blob/master/runners-tests/src/test/java/TearDownTest.java



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3409) Unexpected behavior of DoFn teardown method running in unit tests

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3409?focusedWorklogId=87041&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-87041
 ]

ASF GitHub Bot logged work on BEAM-3409:


Author: ASF GitHub Bot
Created on: 03/Apr/18 11:49
Start Date: 03/Apr/18 11:49
Worklog Time Spent: 10m 
  Work Description: jbonofre commented on issue #4790: [BEAM-3409] 
waitUntilFinish() doesn't wait for the teardown execution on Direct runner - 
fixing compilation issue on flink
URL: https://github.com/apache/beam/pull/4790#issuecomment-378222616
 
 
   Can you please rebase and fix conflicts ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 87041)
Time Spent: 5h 40m  (was: 5.5h)

> Unexpected behavior of DoFn teardown method running in unit tests 
> --
>
> Key: BEAM-3409
> URL: https://issues.apache.org/jira/browse/BEAM-3409
> Project: Beam
>  Issue Type: Bug
>  Components: runner-direct
>Affects Versions: 2.3.0
>Reporter: Alexey Romanenko
>Assignee: Romain Manni-Bucau
>Priority: Blocker
>  Labels: test
> Fix For: 2.5.0
>
>  Time Spent: 5h 40m
>  Remaining Estimate: 0h
>
> Writing a unit test, I found out a strange behaviour of Teardown method of 
> DoFn implementation when I run this method in unit tests using TestPipeline.
> To be more precise, it doesn’t wait until teardown() method will be finished, 
> it just exits from this method after about 1 sec (on my machine) even if it 
> should take longer (very simple example - running infinite loop inside this 
> method or put thread in sleep). In the same time, when I run the same code 
> from main() with ordinary Pipeline and direct runner, then it’s ok and it 
> works as expected - teardown() method will be performed completely despite 
> how much time it will take.
> I created two test cases to reproduce this issue - the first one to run with 
> main() and the second one to run with junit. They use the same implementation 
> of DoFn (class LongTearDownFn) and expects that teardown method will be 
> running at least for SLEEP_TIME ms. In case of running as junit test it's not 
> a case (see output log).
> - run with main()
> https://github.com/aromanenko-dev/beam-samples/blob/master/runners-tests/src/main/java/TearDown.java
> - run with junit
> https://github.com/aromanenko-dev/beam-samples/blob/master/runners-tests/src/test/java/TearDownTest.java



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3409) Unexpected behavior of DoFn teardown method running in unit tests

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3409?focusedWorklogId=84385&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-84385
 ]

ASF GitHub Bot logged work on BEAM-3409:


Author: ASF GitHub Bot
Created on: 26/Mar/18 15:12
Start Date: 26/Mar/18 15:12
Worklog Time Spent: 10m 
  Work Description: rmannibucau commented on issue #4790: [BEAM-3409] 
waitUntilFinish() doesn't wait for the teardown execution on Direct runner - 
fixing compilation issue on flink
URL: https://github.com/apache/beam/pull/4790#issuecomment-376201311
 
 
   up?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 84385)
Time Spent: 5.5h  (was: 5h 20m)

> Unexpected behavior of DoFn teardown method running in unit tests 
> --
>
> Key: BEAM-3409
> URL: https://issues.apache.org/jira/browse/BEAM-3409
> Project: Beam
>  Issue Type: Bug
>  Components: runner-direct
>Affects Versions: 2.3.0
>Reporter: Alexey Romanenko
>Assignee: Romain Manni-Bucau
>Priority: Blocker
>  Labels: test
> Fix For: 2.5.0
>
>  Time Spent: 5.5h
>  Remaining Estimate: 0h
>
> Writing a unit test, I found out a strange behaviour of Teardown method of 
> DoFn implementation when I run this method in unit tests using TestPipeline.
> To be more precise, it doesn’t wait until teardown() method will be finished, 
> it just exits from this method after about 1 sec (on my machine) even if it 
> should take longer (very simple example - running infinite loop inside this 
> method or put thread in sleep). In the same time, when I run the same code 
> from main() with ordinary Pipeline and direct runner, then it’s ok and it 
> works as expected - teardown() method will be performed completely despite 
> how much time it will take.
> I created two test cases to reproduce this issue - the first one to run with 
> main() and the second one to run with junit. They use the same implementation 
> of DoFn (class LongTearDownFn) and expects that teardown method will be 
> running at least for SLEEP_TIME ms. In case of running as junit test it's not 
> a case (see output log).
> - run with main()
> https://github.com/aromanenko-dev/beam-samples/blob/master/runners-tests/src/main/java/TearDown.java
> - run with junit
> https://github.com/aromanenko-dev/beam-samples/blob/master/runners-tests/src/test/java/TearDownTest.java



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3409) Unexpected behavior of DoFn teardown method running in unit tests

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3409?focusedWorklogId=80734&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80734
 ]

ASF GitHub Bot logged work on BEAM-3409:


Author: ASF GitHub Bot
Created on: 15/Mar/18 06:09
Start Date: 15/Mar/18 06:09
Worklog Time Spent: 10m 
  Work Description: rmannibucau commented on issue #4790: [BEAM-3409] 
waitUntilFinish() doesn't wait for the teardown execution on Direct runner - 
fixing compilation issue on flink
URL: https://github.com/apache/beam/pull/4790#issuecomment-373271112
 
 
   R: @jbonofre 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80734)
Time Spent: 5h 20m  (was: 5h 10m)

> Unexpected behavior of DoFn teardown method running in unit tests 
> --
>
> Key: BEAM-3409
> URL: https://issues.apache.org/jira/browse/BEAM-3409
> Project: Beam
>  Issue Type: Bug
>  Components: runner-direct
>Affects Versions: 2.3.0
>Reporter: Alexey Romanenko
>Assignee: Romain Manni-Bucau
>Priority: Blocker
>  Labels: test
> Fix For: 2.5.0
>
>  Time Spent: 5h 20m
>  Remaining Estimate: 0h
>
> Writing a unit test, I found out a strange behaviour of Teardown method of 
> DoFn implementation when I run this method in unit tests using TestPipeline.
> To be more precise, it doesn’t wait until teardown() method will be finished, 
> it just exits from this method after about 1 sec (on my machine) even if it 
> should take longer (very simple example - running infinite loop inside this 
> method or put thread in sleep). In the same time, when I run the same code 
> from main() with ordinary Pipeline and direct runner, then it’s ok and it 
> works as expected - teardown() method will be performed completely despite 
> how much time it will take.
> I created two test cases to reproduce this issue - the first one to run with 
> main() and the second one to run with junit. They use the same implementation 
> of DoFn (class LongTearDownFn) and expects that teardown method will be 
> running at least for SLEEP_TIME ms. In case of running as junit test it's not 
> a case (see output log).
> - run with main()
> https://github.com/aromanenko-dev/beam-samples/blob/master/runners-tests/src/main/java/TearDown.java
> - run with junit
> https://github.com/aromanenko-dev/beam-samples/blob/master/runners-tests/src/test/java/TearDownTest.java



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)