[GitHub] flink pull request #4454: [hotfix][docs] Add section in docs about writing u...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/flink/pull/4454


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #4454: [hotfix][docs] Add section in docs about writing u...

[pnowojski]

Github user pnowojski commented on a diff in the pull request:

https://github.com/apache/flink/pull/4454#discussion_r132112682
  
--- Diff: docs/dev/testing.md ---
@@ -0,0 +1,189 @@
+---
+title: "Testing"
+nav-parent_id: dev
+nav-id: testing
+nav-pos: 99
+---
+
+
+This page briefly discusses how to test Flink application in the local 
environment.
+
+* This will be replaced by the TOC
+{:toc}
+
+## Unit testing
+
+It is encouraged to test your classes with unit tests as much as possible. 
For example if one implement following `ReduceFunction`:
+
+~~~java
+public class SumReduce implements ReduceFunction {
+@Override
+public Long reduce(Long value1, Long value2) throws Exception {
+return value1 + value2;
+}
+}
+~~~
+
+it is very easy to unit test it with your favorite framework:
+
+~~~java
+public class SumReduceTest {
+@Test
+public void testSum() throws Exception {
+SumReduce sumReduce = new SumReduce();
+
+assertEquals(42L, sumReduce.reduce(40L, 2L));
+}
+}
+~~~
+
+Or in scala:
+
+~~~scala
+class SumReduce extends ReduceFunction[Long] {
+override def reduce(value1: java.lang.Long,
+value2: java.lang.Long): java.lang.Long = value1 + 
value2
+}
+~~~
+
+~~~scala
+class SumReduceTest extends FlatSpec with Matchers {
+"SumReduce" should "add values" in {
+val sumReduce: SumReduce = new SumReduce()
+sumReduce.reduce(40L, 2L) should be (42L)
+}
+}
+~~~
+
+## Integration testing
+
+You also can write integration tests that are executed against local Flink 
mini cluster.
+In order to do so add a test dependency `flink-test-utils`.
+
+~~~ xml
+
+  org.apache.flink
+  flink-test-utils{{site.scala_version_suffix}}
+  {{site.version}}
+
+~~~
--- End diff --

Ok, got it. Now it makes sense :)

I have changed this to `{% highlight xml %}` version.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #4454: [hotfix][docs] Add section in docs about writing u...

[dawidwys]

Github user dawidwys commented on a diff in the pull request:

https://github.com/apache/flink/pull/4454#discussion_r131906203
  
--- Diff: docs/dev/testing.md ---
@@ -0,0 +1,189 @@
+---
+title: "Testing"
+nav-parent_id: dev
+nav-id: testing
+nav-pos: 99
+---
+
+
+This page briefly discusses how to test Flink application in the local 
environment.
+
+* This will be replaced by the TOC
+{:toc}
+
+## Unit testing
+
+It is encouraged to test your classes with unit tests as much as possible. 
For example if one implement following `ReduceFunction`:
+
+~~~java
+public class SumReduce implements ReduceFunction {
+@Override
+public Long reduce(Long value1, Long value2) throws Exception {
+return value1 + value2;
+}
+}
+~~~
+
+it is very easy to unit test it with your favorite framework:
+
+~~~java
+public class SumReduceTest {
+@Test
+public void testSum() throws Exception {
+SumReduce sumReduce = new SumReduce();
+
+assertEquals(42L, sumReduce.reduce(40L, 2L));
+}
+}
+~~~
+
+Or in scala:
+
+~~~scala
+class SumReduce extends ReduceFunction[Long] {
+override def reduce(value1: java.lang.Long,
+value2: java.lang.Long): java.lang.Long = value1 + 
value2
+}
+~~~
+
+~~~scala
+class SumReduceTest extends FlatSpec with Matchers {
+"SumReduce" should "add values" in {
+val sumReduce: SumReduce = new SumReduce()
+sumReduce.reduce(40L, 2L) should be (42L)
+}
+}
+~~~
+
+## Integration testing
+
+You also can write integration tests that are executed against local Flink 
mini cluster.
+In order to do so add a test dependency `flink-test-utils`.
+
+~~~ xml
+
+  org.apache.flink
+  flink-test-utils{{site.scala_version_suffix}}
+  {{site.version}}
+
+~~~
--- End diff --

Just a short explanation. In other places e.g. in `dev/connectors/kafka.md` 
we use other highlighting extension(liquid) and a code like:

{% highlight xml %}

  org.apache.flink
  flink-test-utils{{ site.scala_version_suffix 
}}
  {{site.version }}

{% endhighlight %}

renders into:


  org.apache.flink
  flink-test-utils_2.10
  1.4-SNAPSHOT


`{{site.version }}` and `{{ site.scala_version_suffix }}` are variables 
based on properties of the build. It seems they do not work with the syntax 
used in this site(that uses redcarpet highlighting).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #4454: [hotfix][docs] Add section in docs about writing u...

[pnowojski]

Github user pnowojski commented on a diff in the pull request:

https://github.com/apache/flink/pull/4454#discussion_r131900306
  
--- Diff: docs/dev/testing.md ---
@@ -0,0 +1,189 @@
+---
+title: "Testing"
+nav-parent_id: dev
+nav-id: testing
+nav-pos: 99
+---
+
+
+This page briefly discusses how to test Flink application in the local 
environment.
+
+* This will be replaced by the TOC
+{:toc}
+
+## Unit testing
+
+It is encouraged to test your classes with unit tests as much as possible. 
For example if one implement following `ReduceFunction`:
+
+~~~java
+public class SumReduce implements ReduceFunction {
+@Override
+public Long reduce(Long value1, Long value2) throws Exception {
+return value1 + value2;
+}
+}
+~~~
+
+it is very easy to unit test it with your favorite framework:
+
+~~~java
+public class SumReduceTest {
+@Test
+public void testSum() throws Exception {
+SumReduce sumReduce = new SumReduce();
+
+assertEquals(42L, sumReduce.reduce(40L, 2L));
+}
+}
+~~~
+
+Or in scala:
+
+~~~scala
+class SumReduce extends ReduceFunction[Long] {
+override def reduce(value1: java.lang.Long,
+value2: java.lang.Long): java.lang.Long = value1 + 
value2
+}
+~~~
+
+~~~scala
+class SumReduceTest extends FlatSpec with Matchers {
+"SumReduce" should "add values" in {
+val sumReduce: SumReduce = new SumReduce()
+sumReduce.reduce(40L, 2L) should be (42L)
+}
+}
+~~~
+
+## Integration testing
+
+You also can write integration tests that are executed against local Flink 
mini cluster.
+In order to do so add a test dependency `flink-test-utils`.
+
+~~~ xml
+
+  org.apache.flink
+  flink-test-utils{{site.scala_version_suffix}}
+  {{site.version}}
+
+~~~
--- End diff --

Updated but it properties names depends on the user's `pom.xml` file.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #4454: [hotfix][docs] Add section in docs about writing u...

[alpinegizmo]

Github user alpinegizmo commented on a diff in the pull request:

https://github.com/apache/flink/pull/4454#discussion_r131354369
  
--- Diff: docs/dev/testing.md ---
@@ -0,0 +1,189 @@
+---
+title: "Testing"
+nav-parent_id: dev
+nav-id: testing
+nav-pos: 99
+---
+
+
+This page briefly discusses how to test Flink application in the local 
environment.
+
+* This will be replaced by the TOC
+{:toc}
+
+## Unit testing
+
+It is encouraged to test your classes with unit tests as much as possible. 
For example if one implement following `ReduceFunction`:
+
+~~~java
+public class SumReduce implements ReduceFunction {
+@Override
+public Long reduce(Long value1, Long value2) throws Exception {
+return value1 + value2;
+}
+}
+~~~
+
+it is very easy to unit test it with your favorite framework:
+
+~~~java
+public class SumReduceTest {
+@Test
+public void testSum() throws Exception {
+SumReduce sumReduce = new SumReduce();
+
+assertEquals(42L, sumReduce.reduce(40L, 2L));
+}
+}
+~~~
+
+Or in scala:
+
+~~~scala
+class SumReduce extends ReduceFunction[Long] {
+override def reduce(value1: java.lang.Long,
+value2: java.lang.Long): java.lang.Long = value1 + 
value2
+}
+~~~
+
+~~~scala
+class SumReduceTest extends FlatSpec with Matchers {
+"SumReduce" should "add values" in {
+val sumReduce: SumReduce = new SumReduce()
+sumReduce.reduce(40L, 2L) should be (42L)
+}
+}
+~~~
+
+## Integration testing
+
+You also can write integration tests that are executed against local Flink 
mini cluster.
+In order to do so add a test dependency `flink-test-utils`.
+
+~~~ xml
+
+  org.apache.flink
+  flink-test-utils{{site.scala_version_suffix}}
+  {{site.version}}
+
+~~~
--- End diff --

To get this to work in my outside-of-flink project, I had to modify this as:


  org.apache.flink
  
flink-test-utils_{{site.scala_version_suffix}}
  ${flink.version}
 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #4454: [hotfix][docs] Add section in docs about writing u...

[alpinegizmo]

Github user alpinegizmo commented on a diff in the pull request:

https://github.com/apache/flink/pull/4454#discussion_r131352677
  
--- Diff: docs/dev/testing.md ---
@@ -90,13 +121,69 @@ public class ExampleIntegrationTest extends 
StreamingMultipleProgramsTestBase {
 public static final List values = new ArrayList<>();
 
 @Override
-public void invoke(Long value) throws Exception {
+public synchronized void invoke(Long value) throws Exception {
 values.add(value);
 }
 }
 }
 ~~~
 
-Static variable in `CollectSink` is required because Flink serializes all 
operators before distributing them across a cluster.
+or in Scala:
+
+~~~scala
+class MultiplyByTwo extends MapFunction[Long, Long] {
+  override def map(value: java.lang.Long): java.lang.Long = value * 2
+}
+~~~
+
+~~~scala
+class ExampleIntegrationTest extends FlatSpec with Matchers {
+"MultiplyByTwo" should "multiply it input by two" in {
+val env: StreamExecutionEnvironment =
+StreamExecutionEnvironment.getExecutionEnvironment
+env.setParallelism(1)
+// values are collected on a static variable
+CollectSink.values.clear()
+env
+.fromElements(1L, 21L, 22L)
+.map(new MultiplyByTwo())
+.addSink(new CollectSink())
+env.execute()
+CollectSink.values should be (Lists.newArrayList(2L, 42L, 44L))
+}
+}
+
+object CollectSink {
+// must be static
+val values: List[Long] = new ArrayList()
+}
+
+class CollectSink extends SinkFunction[Long] {
+override def invoke(value: java.lang.Long): Unit = {
+synchronized {
+values.add(value)
+}
+}
+}
+~~~
+
+Static variable in `CollectSink` is used here because Flink serializes all 
operators before distributing them across a cluster.
+Communicating with operators instantiated by a local flink mini cluster 
via static variables is one way around this issue.
 Alternatively in your test sink you could for example write the data to 
files in a temporary directory.
 Of course you could use your own custom sources and sinks, which can emit 
watermarks.
+
+## Testing checkpointing and state handling
+
+One way to test state handling is to enable checkpointing in integration 
tests. You can do that by
+configuring `environment` in the test:
+~~~java
+env.enableCheckpointing(500);
+env.setRestartStrategy(RestartStrategies.fixedDelayRestart(3, 100));
+~~~
+and for example adding to your Flink application an identity mapper 
operator that will throw and exception
--- End diff --

"throw an exception" (typo)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #4454: [hotfix][docs] Add section in docs about writing u...

[pnowojski]

Github user pnowojski commented on a diff in the pull request:

https://github.com/apache/flink/pull/4454#discussion_r131343119
  
--- Diff: docs/dev/testing.md ---
@@ -0,0 +1,102 @@
+---
+title: "Testing"
+nav-parent_id: dev
+nav-pos: 110
+---
+
+
+This page briefly discusses how to test Flink application in the local 
environment.
+
+* This will be replaced by the TOC
+{:toc}
+
+## Unit testing
+
+It is encouraged to test your classes with unit tests as much as possible. 
For example if one implement following `ReduceFunction`:
+~~~java
+public class SumReduce implements ReduceFunction {
+@Override
+public Long reduce(Long value1, Long value2) throws Exception {
+return value1 + value2;
+}
+}
+~~~
+it is very easy to unit test it with your favorite framework:
+~~~java
+public class SumReduceTest {
+@Test
+public void testSum() throws Exception {
+SumReduce sumReduce = new SumReduce();
+
+assertEquals(42L, sumReduce.reduce(40L, 2L));
+}
+}
+~~~
+
+## Integration testing
+
+You also can write integration tests that are executed against local Flink 
mini cluster.
+In order to do so add a test dependency `flink-test-utils`. For example if 
you want to
+test following `MapFunction`:
+
+~~~java
+public class MultiplyByTwo implements MapFunction {
+@Override
+public Long map(Long value) throws Exception {
+return value * 2;
+}
+}
+~~~
+
+You could write following integration test:
+
+~~~java
+public class ExampleIntegrationTest extends 
StreamingMultipleProgramsTestBase {
+@Test
+public void testSum() throws Exception {
+StreamExecutionEnvironment env = 
StreamExecutionEnvironment.getExecutionEnvironment();
+env.setParallelism(1);
+
+// values are collected on a static variable
+CollectSink.values.clear();
+
+env.fromElements(1L, 21L, 22L)
+.map(new MultiplyByTwo())
+.addSink(new CollectSink());
+env.execute();
+
+assertEquals(Lists.newArrayList(2L, 42L, 44L), CollectSink.values);
+}
+
+private static class CollectSink implements SinkFunction {
+// must be static
+public static final List values = new ArrayList<>();
+
+@Override
+public void invoke(Long value) throws Exception {
+values.add(value);
--- End diff --

It depends on the parallelism, but ok, it will be safer to synchronize it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #4454: [hotfix][docs] Add section in docs about writing u...

[pnowojski]

Github user pnowojski commented on a diff in the pull request:

https://github.com/apache/flink/pull/4454#discussion_r131337294
  
--- Diff: docs/dev/testing.md ---
@@ -0,0 +1,102 @@
+---
+title: "Testing"
+nav-parent_id: dev
+nav-pos: 110
+---
+
+
+This page briefly discusses how to test Flink application in the local 
environment.
+
+* This will be replaced by the TOC
+{:toc}
+
+## Unit testing
+
+It is encouraged to test your classes with unit tests as much as possible. 
For example if one implement following `ReduceFunction`:
+~~~java
+public class SumReduce implements ReduceFunction {
+@Override
+public Long reduce(Long value1, Long value2) throws Exception {
+return value1 + value2;
+}
+}
+~~~
+it is very easy to unit test it with your favorite framework:
--- End diff --

I don't see a reason why.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #4454: [hotfix][docs] Add section in docs about writing u...

[pnowojski]

Github user pnowojski commented on a diff in the pull request:

https://github.com/apache/flink/pull/4454#discussion_r131344197
  
--- Diff: docs/dev/testing.md ---
@@ -0,0 +1,102 @@
+---
+title: "Testing"
+nav-parent_id: dev
+nav-pos: 110
+---
+
+
+This page briefly discusses how to test Flink application in the local 
environment.
+
+* This will be replaced by the TOC
+{:toc}
+
+## Unit testing
+
+It is encouraged to test your classes with unit tests as much as possible. 
For example if one implement following `ReduceFunction`:
+~~~java
+public class SumReduce implements ReduceFunction {
+@Override
+public Long reduce(Long value1, Long value2) throws Exception {
+return value1 + value2;
+}
+}
+~~~
+it is very easy to unit test it with your favorite framework:
+~~~java
+public class SumReduceTest {
+@Test
+public void testSum() throws Exception {
+SumReduce sumReduce = new SumReduce();
+
+assertEquals(42L, sumReduce.reduce(40L, 2L));
+}
+}
+~~~
+
+## Integration testing
+
+You also can write integration tests that are executed against local Flink 
mini cluster.
+In order to do so add a test dependency `flink-test-utils`. For example if 
you want to
+test following `MapFunction`:
+
+~~~java
+public class MultiplyByTwo implements MapFunction {
+@Override
+public Long map(Long value) throws Exception {
+return value * 2;
+}
+}
+~~~
+
+You could write following integration test:
+
+~~~java
+public class ExampleIntegrationTest extends 
StreamingMultipleProgramsTestBase {
+@Test
+public void testSum() throws Exception {
+StreamExecutionEnvironment env = 
StreamExecutionEnvironment.getExecutionEnvironment();
+env.setParallelism(1);
+
+// values are collected on a static variable
+CollectSink.values.clear();
+
+env.fromElements(1L, 21L, 22L)
+.map(new MultiplyByTwo())
+.addSink(new CollectSink());
+env.execute();
+
+assertEquals(Lists.newArrayList(2L, 42L, 44L), CollectSink.values);
+}
+
+private static class CollectSink implements SinkFunction {
+// must be static
+public static final List values = new ArrayList<>();
+
+@Override
+public void invoke(Long value) throws Exception {
+values.add(value);
+}
+}
+}
+~~~
+
+Static variable in `CollectSink` is required because Flink serializes all 
operators before distributing them across a cluster.
--- End diff --

I don't want to drop it, because such construct with `static` fields used 
in test was a little bit confusing for me when I started working with Flink.

I have rephrased this section a little bit.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #4454: [hotfix][docs] Add section in docs about writing u...

[pnowojski]

Github user pnowojski commented on a diff in the pull request:

https://github.com/apache/flink/pull/4454#discussion_r131342944
  
--- Diff: docs/dev/testing.md ---
@@ -0,0 +1,102 @@
+---
+title: "Testing"
+nav-parent_id: dev
+nav-pos: 110
+---
+
+
+This page briefly discusses how to test Flink application in the local 
environment.
+
+* This will be replaced by the TOC
+{:toc}
+
+## Unit testing
+
+It is encouraged to test your classes with unit tests as much as possible. 
For example if one implement following `ReduceFunction`:
+~~~java
--- End diff --

bah... Done, that were my first Scala lines written in my life


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #4454: [hotfix][docs] Add section in docs about writing u...

[twalthr]

Github user twalthr commented on a diff in the pull request:

https://github.com/apache/flink/pull/4454#discussion_r131100075
  
--- Diff: docs/dev/testing.md ---
@@ -0,0 +1,102 @@
+---
+title: "Testing"
+nav-parent_id: dev
+nav-pos: 110
+---
+
+
+This page briefly discusses how to test Flink application in the local 
environment.
+
+* This will be replaced by the TOC
+{:toc}
+
+## Unit testing
+
+It is encouraged to test your classes with unit tests as much as possible. 
For example if one implement following `ReduceFunction`:
+~~~java
--- End diff --

Usually, we implement code examples in both Scala and Java.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #4454: [hotfix][docs] Add section in docs about writing u...

[twalthr]

Github user twalthr commented on a diff in the pull request:

https://github.com/apache/flink/pull/4454#discussion_r131099750
  
--- Diff: docs/dev/testing.md ---
@@ -0,0 +1,102 @@
+---
+title: "Testing"
+nav-parent_id: dev
+nav-pos: 110
+---
+
+
+This page briefly discusses how to test Flink application in the local 
environment.
+
+* This will be replaced by the TOC
+{:toc}
+
+## Unit testing
+
+It is encouraged to test your classes with unit tests as much as possible. 
For example if one implement following `ReduceFunction`:
+~~~java
+public class SumReduce implements ReduceFunction {
+@Override
+public Long reduce(Long value1, Long value2) throws Exception {
+return value1 + value2;
+}
+}
+~~~
+it is very easy to unit test it with your favorite framework:
+~~~java
+public class SumReduceTest {
+@Test
+public void testSum() throws Exception {
+SumReduce sumReduce = new SumReduce();
+
+assertEquals(42L, sumReduce.reduce(40L, 2L));
+}
+}
+~~~
+
+## Integration testing
+
+You also can write integration tests that are executed against local Flink 
mini cluster.
+In order to do so add a test dependency `flink-test-utils`. For example if 
you want to
+test following `MapFunction`:
+
+~~~java
+public class MultiplyByTwo implements MapFunction {
+@Override
+public Long map(Long value) throws Exception {
+return value * 2;
+}
+}
+~~~
+
+You could write following integration test:
+
+~~~java
+public class ExampleIntegrationTest extends 
StreamingMultipleProgramsTestBase {
+@Test
+public void testSum() throws Exception {
+StreamExecutionEnvironment env = 
StreamExecutionEnvironment.getExecutionEnvironment();
+env.setParallelism(1);
+
+// values are collected on a static variable
+CollectSink.values.clear();
+
+env.fromElements(1L, 21L, 22L)
+.map(new MultiplyByTwo())
+.addSink(new CollectSink());
+env.execute();
+
+assertEquals(Lists.newArrayList(2L, 42L, 44L), CollectSink.values);
+}
+
+private static class CollectSink implements SinkFunction {
+// must be static
+public static final List values = new ArrayList<>();
+
+@Override
+public void invoke(Long value) throws Exception {
+values.add(value);
--- End diff --

Does this function has to be synchronized? At least we are doing it in 
`org.apache.flink.table.runtime.utils.StreamITCase`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #4454: [hotfix][docs] Add section in docs about writing u...

[tzulitai]

Github user tzulitai commented on a diff in the pull request:

https://github.com/apache/flink/pull/4454#discussion_r131060102
  
--- Diff: docs/dev/testing.md ---
@@ -0,0 +1,102 @@
+---
+title: "Testing"
+nav-parent_id: dev
+nav-pos: 110
+---
+
+
+This page briefly discusses how to test Flink application in the local 
environment.
+
+* This will be replaced by the TOC
+{:toc}
+
+## Unit testing
+
+It is encouraged to test your classes with unit tests as much as possible. 
For example if one implement following `ReduceFunction`:
+~~~java
+public class SumReduce implements ReduceFunction {
+@Override
+public Long reduce(Long value1, Long value2) throws Exception {
+return value1 + value2;
+}
+}
+~~~
+it is very easy to unit test it with your favorite framework:
+~~~java
+public class SumReduceTest {
+@Test
+public void testSum() throws Exception {
+SumReduce sumReduce = new SumReduce();
+
+assertEquals(42L, sumReduce.reduce(40L, 2L));
+}
+}
+~~~
+
+## Integration testing
+
+You also can write integration tests that are executed against local Flink 
mini cluster.
+In order to do so add a test dependency `flink-test-utils`. For example if 
you want to
+test following `MapFunction`:
+
+~~~java
+public class MultiplyByTwo implements MapFunction {
+@Override
+public Long map(Long value) throws Exception {
+return value * 2;
+}
+}
+~~~
+
+You could write following integration test:
+
+~~~java
+public class ExampleIntegrationTest extends 
StreamingMultipleProgramsTestBase {
+@Test
+public void testSum() throws Exception {
+StreamExecutionEnvironment env = 
StreamExecutionEnvironment.getExecutionEnvironment();
+env.setParallelism(1);
+
+// values are collected on a static variable
+CollectSink.values.clear();
+
+env.fromElements(1L, 21L, 22L)
+.map(new MultiplyByTwo())
+.addSink(new CollectSink());
+env.execute();
+
+assertEquals(Lists.newArrayList(2L, 42L, 44L), CollectSink.values);
+}
+
+private static class CollectSink implements SinkFunction {
+// must be static
+public static final List values = new ArrayList<>();
+
+@Override
+public void invoke(Long value) throws Exception {
+values.add(value);
+}
+}
+}
+~~~
+
+Static variable in `CollectSink` is required because Flink serializes all 
operators before distributing them across a cluster.
--- End diff --

I find this instruction a bit too specific. There's other ways to achieve 
this, such as just initializing it in the open() method.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #4454: [hotfix][docs] Add section in docs about writing u...

[tzulitai]

Github user tzulitai commented on a diff in the pull request:

https://github.com/apache/flink/pull/4454#discussion_r131059884
  
--- Diff: docs/dev/testing.md ---
@@ -0,0 +1,102 @@
+---
+title: "Testing"
+nav-parent_id: dev
+nav-pos: 110
+---
+
+
+This page briefly discusses how to test Flink application in the local 
environment.
+
+* This will be replaced by the TOC
+{:toc}
+
+## Unit testing
+
+It is encouraged to test your classes with unit tests as much as possible. 
For example if one implement following `ReduceFunction`:
+~~~java
+public class SumReduce implements ReduceFunction {
+@Override
+public Long reduce(Long value1, Long value2) throws Exception {
+return value1 + value2;
+}
+}
+~~~
+it is very easy to unit test it with your favorite framework:
--- End diff --

question aout "favorite framework":
do we actually want to suggest a framework and be more specific here?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #4454: [hotfix][docs] Add section in docs about writing u...

[tzulitai]

Github user tzulitai commented on a diff in the pull request:

https://github.com/apache/flink/pull/4454#discussion_r131059770
  
--- Diff: docs/dev/testing.md ---
@@ -0,0 +1,102 @@
+---
+title: "Testing"
+nav-parent_id: dev
+nav-pos: 110
+---
+
+
+This page briefly discusses how to test Flink application in the local 
environment.
+
+* This will be replaced by the TOC
+{:toc}
+
+## Unit testing
+
+It is encouraged to test your classes with unit tests as much as possible. 
For example if one implement following `ReduceFunction`:
+~~~java
+public class SumReduce implements ReduceFunction {
+@Override
+public Long reduce(Long value1, Long value2) throws Exception {
+return value1 + value2;
+}
+}
+~~~
+it is very easy to unit test it with your favorite framework:
+~~~java
+public class SumReduceTest {
+@Test
+public void testSum() throws Exception {
+SumReduce sumReduce = new SumReduce();
+
+assertEquals(42L, sumReduce.reduce(40L, 2L));
+}
+}
+~~~
+
+## Integration testing
+
+You also can write integration tests that are executed against local Flink 
mini cluster.
+In order to do so add a test dependency `flink-test-utils`. For example if 
you want to
--- End diff --

+1 to Dawid's suggestion. Usually it's helpful to be more verbose on these 
procedures.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #4454: [hotfix][docs] Add section in docs about writing u...

[dawidwys]

Github user dawidwys commented on a diff in the pull request:

https://github.com/apache/flink/pull/4454#discussion_r130826551
  
--- Diff: docs/dev/testing.md ---
@@ -0,0 +1,102 @@
+---
+title: "Testing"
+nav-parent_id: dev
+nav-pos: 110
+---
+
+
+This page briefly discusses how to test Flink application in the local 
environment.
+
+* This will be replaced by the TOC
+{:toc}
+
+## Unit testing
+
+It is encouraged to test your classes with unit tests as much as possible. 
For example if one implement following `ReduceFunction`:
+~~~java
+public class SumReduce implements ReduceFunction {
+@Override
+public Long reduce(Long value1, Long value2) throws Exception {
+return value1 + value2;
+}
+}
+~~~
+it is very easy to unit test it with your favorite framework:
+~~~java
+public class SumReduceTest {
+@Test
+public void testSum() throws Exception {
+SumReduce sumReduce = new SumReduce();
+
+assertEquals(42L, sumReduce.reduce(40L, 2L));
+}
+}
+~~~
+
+## Integration testing
+
+You also can write integration tests that are executed against local Flink 
mini cluster.
+In order to do so add a test dependency `flink-test-utils`. For example if 
you want to
--- End diff --

I think you could provide here a full maven dependency code like:

{% highlight xml %}

  org.apache.flink
  flink-test-utils{{ site.scala_version_suffix 
}}
  {{site.version }}

{% endhighlight %}


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #4454: [hotfix][docs] Add section in docs about writing u...

[pnowojski]

GitHub user pnowojski opened a pull request:

https://github.com/apache/flink/pull/4454

[hotfix][docs] Add section in docs about writing unit/integration tests

Improved documentation. CC @twalthr 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/pnowojski/flink docs

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4454.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4454


commit 8c211b8fd975af441c5a762ee45a62a7dd44f173
Author: Piotr Nowojski 
Date:   2017-08-01T13:02:56Z

[hotfix][docs] Add section in docs about writing unit and integration tests




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---