[jira] [Commented] (FLINK-6830) Add ITTests for savepoint migration from 1.3

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16041169#comment-16041169
 ] 

ASF GitHub Bot commented on FLINK-6830:
---

Github user asfgit closed the pull request at:

https://github.com/apache/flink/pull/4059


> Add ITTests for savepoint migration from 1.3
> 
>
> Key: FLINK-6830
> URL: https://issues.apache.org/jira/browse/FLINK-6830
> Project: Flink
>  Issue Type: Test
>  Components: State Backends, Checkpointing
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
> Fix For: 1.3.1
>
>
> Already with FLINK-6763 and FLINK-6764 we'll need to change the serialization 
> formats between 1.3.0 and 1.3.x.
> We probably should add the stateful job migration ITCases for restoring from 
> Flink 1.3.x now.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-6830) Add ITTests for savepoint migration from 1.3

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040676#comment-16040676
 ] 

ASF GitHub Bot commented on FLINK-6830:
---

Github user tzulitai commented on the issue:

https://github.com/apache/flink/pull/4059
  
Merging ...


> Add ITTests for savepoint migration from 1.3
> 
>
> Key: FLINK-6830
> URL: https://issues.apache.org/jira/browse/FLINK-6830
> Project: Flink
>  Issue Type: Test
>  Components: State Backends, Checkpointing
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
> Fix For: 1.3.1
>
>
> Already with FLINK-6763 and FLINK-6764 we'll need to change the serialization 
> formats between 1.3.0 and 1.3.x.
> We probably should add the stateful job migration ITCases for restoring from 
> Flink 1.3.x now.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-6830) Add ITTests for savepoint migration from 1.3

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16039034#comment-16039034
 ] 

ASF GitHub Bot commented on FLINK-6830:
---

Github user tzulitai commented on the issue:

https://github.com/apache/flink/pull/4059
  
Addressed @aljoscha @kl0u @zentol's comments.

Also rebased on `master` for another Travis run. If green, will merge by 
the end of today to `master` and `release-1.3` if there's no other objections!


> Add ITTests for savepoint migration from 1.3
> 
>
> Key: FLINK-6830
> URL: https://issues.apache.org/jira/browse/FLINK-6830
> Project: Flink
>  Issue Type: Test
>  Components: State Backends, Checkpointing
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
> Fix For: 1.3.1
>
>
> Already with FLINK-6763 and FLINK-6764 we'll need to change the serialization 
> formats between 1.3.0 and 1.3.x.
> We probably should add the stateful job migration ITCases for restoring from 
> Flink 1.3.x now.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-6830) Add ITTests for savepoint migration from 1.3

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038953#comment-16038953
 ] 

ASF GitHub Bot commented on FLINK-6830:
---

Github user kl0u commented on the issue:

https://github.com/apache/flink/pull/4059
  
@tzulitai yes I will ;)


> Add ITTests for savepoint migration from 1.3
> 
>
> Key: FLINK-6830
> URL: https://issues.apache.org/jira/browse/FLINK-6830
> Project: Flink
>  Issue Type: Test
>  Components: State Backends, Checkpointing
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
> Fix For: 1.3.1
>
>
> Already with FLINK-6763 and FLINK-6764 we'll need to change the serialization 
> formats between 1.3.0 and 1.3.x.
> We probably should add the stateful job migration ITCases for restoring from 
> Flink 1.3.x now.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-6830) Add ITTests for savepoint migration from 1.3

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038921#comment-16038921
 ] 

ASF GitHub Bot commented on FLINK-6830:
---

Github user tzulitai commented on the issue:

https://github.com/apache/flink/pull/4059
  
Thanks @kl0u, your suggestions fixes the problem. Would it also be possible 
to include document the contiguity semantic changes in #4041?


> Add ITTests for savepoint migration from 1.3
> 
>
> Key: FLINK-6830
> URL: https://issues.apache.org/jira/browse/FLINK-6830
> Project: Flink
>  Issue Type: Test
>  Components: State Backends, Checkpointing
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
> Fix For: 1.3.1
>
>
> Already with FLINK-6763 and FLINK-6764 we'll need to change the serialization 
> formats between 1.3.0 and 1.3.x.
> We probably should add the stateful job migration ITCases for restoring from 
> Flink 1.3.x now.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-6830) Add ITTests for savepoint migration from 1.3

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038918#comment-16038918
 ] 

ASF GitHub Bot commented on FLINK-6830:
---

Github user tzulitai commented on a diff in the pull request:

https://github.com/apache/flink/pull/4059#discussion_r120365906
  
--- Diff: 
flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/operators/windowing/WindowOperatorMigrationTest.java
 ---
@@ -149,9 +169,16 @@ public void 
testRestoreSessionWindowsWithCountTrigger() throws Exception {
new 
KeyedOneInputStreamOperatorTestHarness<>(operator, new TupleKeySelector(), 
BasicTypeInfo.STRING_TYPE_INFO);
 
testHarness.setup();
-   testHarness.initializeState(
-   OperatorSnapshotUtil.readStateHandle(
-   
OperatorSnapshotUtil.getResourceFilename("win-op-migration-test-session-with-stateful-trigger-flink1.2-snapshot")));
+
+   String savepointFile = 
"win-op-migration-test-session-with-stateful-trigger-flink" + 
testMigrateVersion + "-snapshot";
+   if (testMigrateVersion.equals("1.1")) {
--- End diff --

That makes sense! I'll add the method to some `MigrationTestUtil` perhaps.


> Add ITTests for savepoint migration from 1.3
> 
>
> Key: FLINK-6830
> URL: https://issues.apache.org/jira/browse/FLINK-6830
> Project: Flink
>  Issue Type: Test
>  Components: State Backends, Checkpointing
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
> Fix For: 1.3.1
>
>
> Already with FLINK-6763 and FLINK-6764 we'll need to change the serialization 
> formats between 1.3.0 and 1.3.x.
> We probably should add the stateful job migration ITCases for restoring from 
> Flink 1.3.x now.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-6830) Add ITTests for savepoint migration from 1.3

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038859#comment-16038859
 ] 

ASF GitHub Bot commented on FLINK-6830:
---

Github user kl0u commented on the issue:

https://github.com/apache/flink/pull/4059
  
Hi @tzulitai ! Thanks for testing also this. 

The problem with the `CEP` failing test for `1.3` while passing for `1.2` 
is that the semantics of the `followedBy()` changed between `1.2` and `1.3`, so 
the expected results cannot be the same. 

In `1.2`, `followedBy()` implied non-deterministic relaxed contiguity while 
in `1.3` it is just relaxed contiguity. So for `1.3`, either change the 
expected result to contain only one match with the `foo1` as middle element, or 
change the pattern to have `followedByAny(middle)` (and in this case you have 
to take the savepoint again).


> Add ITTests for savepoint migration from 1.3
> 
>
> Key: FLINK-6830
> URL: https://issues.apache.org/jira/browse/FLINK-6830
> Project: Flink
>  Issue Type: Test
>  Components: State Backends, Checkpointing
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
> Fix For: 1.3.1
>
>
> Already with FLINK-6763 and FLINK-6764 we'll need to change the serialization 
> formats between 1.3.0 and 1.3.x.
> We probably should add the stateful job migration ITCases for restoring from 
> Flink 1.3.x now.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-6830) Add ITTests for savepoint migration from 1.3

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038556#comment-16038556
 ] 

ASF GitHub Bot commented on FLINK-6830:
---

Github user aljoscha commented on the issue:

https://github.com/apache/flink/pull/4059
  
I like these changes a lot! Especially the refactoring of using the same 
test for all versions. (I'm sure @zentol will also like that  )

Had some inline comments.


> Add ITTests for savepoint migration from 1.3
> 
>
> Key: FLINK-6830
> URL: https://issues.apache.org/jira/browse/FLINK-6830
> Project: Flink
>  Issue Type: Test
>  Components: State Backends, Checkpointing
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
> Fix For: 1.3.1
>
>
> Already with FLINK-6763 and FLINK-6764 we'll need to change the serialization 
> formats between 1.3.0 and 1.3.x.
> We probably should add the stateful job migration ITCases for restoring from 
> Flink 1.3.x now.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-6830) Add ITTests for savepoint migration from 1.3

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038555#comment-16038555
 ] 

ASF GitHub Bot commented on FLINK-6830:
---

Github user aljoscha commented on a diff in the pull request:

https://github.com/apache/flink/pull/4059#discussion_r120317328
  
--- Diff: 
flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/operators/windowing/WindowOperatorMigrationTest.java
 ---
@@ -149,9 +169,16 @@ public void 
testRestoreSessionWindowsWithCountTrigger() throws Exception {
new 
KeyedOneInputStreamOperatorTestHarness<>(operator, new TupleKeySelector(), 
BasicTypeInfo.STRING_TYPE_INFO);
 
testHarness.setup();
-   testHarness.initializeState(
-   OperatorSnapshotUtil.readStateHandle(
-   
OperatorSnapshotUtil.getResourceFilename("win-op-migration-test-session-with-stateful-trigger-flink1.2-snapshot")));
+
+   String savepointFile = 
"win-op-migration-test-session-with-stateful-trigger-flink" + 
testMigrateVersion + "-snapshot";
+   if (testMigrateVersion.equals("1.1")) {
--- End diff --

I think you could refactor this piece of code into a method, something like:
```
restoreFromSnapshot(TestHarness, SnapshotPath, MigrationVersion)
```

What do you think?


> Add ITTests for savepoint migration from 1.3
> 
>
> Key: FLINK-6830
> URL: https://issues.apache.org/jira/browse/FLINK-6830
> Project: Flink
>  Issue Type: Test
>  Components: State Backends, Checkpointing
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
> Fix For: 1.3.1
>
>
> Already with FLINK-6763 and FLINK-6764 we'll need to change the serialization 
> formats between 1.3.0 and 1.3.x.
> We probably should add the stateful job migration ITCases for restoring from 
> Flink 1.3.x now.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-6830) Add ITTests for savepoint migration from 1.3

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038544#comment-16038544
 ] 

ASF GitHub Bot commented on FLINK-6830:
---

Github user kl0u commented on the issue:

https://github.com/apache/flink/pull/4059
  
@aljoscha yes we have already discussed offline with @tzulitai .


> Add ITTests for savepoint migration from 1.3
> 
>
> Key: FLINK-6830
> URL: https://issues.apache.org/jira/browse/FLINK-6830
> Project: Flink
>  Issue Type: Test
>  Components: State Backends, Checkpointing
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
> Fix For: 1.3.1
>
>
> Already with FLINK-6763 and FLINK-6764 we'll need to change the serialization 
> formats between 1.3.0 and 1.3.x.
> We probably should add the stateful job migration ITCases for restoring from 
> Flink 1.3.x now.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-6830) Add ITTests for savepoint migration from 1.3

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038503#comment-16038503
 ] 

ASF GitHub Bot commented on FLINK-6830:
---

Github user aljoscha commented on the issue:

https://github.com/apache/flink/pull/4059
  
@kl0u Could you please look into the CEP tests that failing to restore from 
1.3 to 1.3?


> Add ITTests for savepoint migration from 1.3
> 
>
> Key: FLINK-6830
> URL: https://issues.apache.org/jira/browse/FLINK-6830
> Project: Flink
>  Issue Type: Test
>  Components: State Backends, Checkpointing
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
> Fix For: 1.3.1
>
>
> Already with FLINK-6763 and FLINK-6764 we'll need to change the serialization 
> formats between 1.3.0 and 1.3.x.
> We probably should add the stateful job migration ITCases for restoring from 
> Flink 1.3.x now.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-6830) Add ITTests for savepoint migration from 1.3

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038497#comment-16038497
 ] 

ASF GitHub Bot commented on FLINK-6830:
---

Github user aljoscha commented on the issue:

https://github.com/apache/flink/pull/4059
  
Why are the CEP "from12" snapshots also changed?


> Add ITTests for savepoint migration from 1.3
> 
>
> Key: FLINK-6830
> URL: https://issues.apache.org/jira/browse/FLINK-6830
> Project: Flink
>  Issue Type: Test
>  Components: State Backends, Checkpointing
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
> Fix For: 1.3.1
>
>
> Already with FLINK-6763 and FLINK-6764 we'll need to change the serialization 
> formats between 1.3.0 and 1.3.x.
> We probably should add the stateful job migration ITCases for restoring from 
> Flink 1.3.x now.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-6830) Add ITTests for savepoint migration from 1.3

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038461#comment-16038461
 ] 

ASF GitHub Bot commented on FLINK-6830:
---

Github user tzulitai commented on a diff in the pull request:

https://github.com/apache/flink/pull/4059#discussion_r120305797
  
--- Diff: 
flink-tests/src/test/java/org/apache/flink/test/state/operator/restore/unkeyed/ChainBreakTest.java
 ---
@@ -31,8 +36,22 @@
 /**
  * Verifies that the state of all operators is restored if a topology 
change breaks up a chain.
  */
+@RunWith(Parameterized.class)
 public class ChainBreakTest extends 
AbstractNonKeyedOperatorRestoreTestBase {
 
+   private final String savepointPath;
+
+   @Parameterized.Parameters(name = "Migrate Savepoint: {0}")
+   public static Collection parameters () {
--- End diff --

Will change!


> Add ITTests for savepoint migration from 1.3
> 
>
> Key: FLINK-6830
> URL: https://issues.apache.org/jira/browse/FLINK-6830
> Project: Flink
>  Issue Type: Test
>  Components: State Backends, Checkpointing
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
> Fix For: 1.3.1
>
>
> Already with FLINK-6763 and FLINK-6764 we'll need to change the serialization 
> formats between 1.3.0 and 1.3.x.
> We probably should add the stateful job migration ITCases for restoring from 
> Flink 1.3.x now.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-6830) Add ITTests for savepoint migration from 1.3

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038444#comment-16038444
 ] 

ASF GitHub Bot commented on FLINK-6830:
---

Github user zentol commented on the issue:

https://github.com/apache/flink/pull/4059
  
topology tests look good.


> Add ITTests for savepoint migration from 1.3
> 
>
> Key: FLINK-6830
> URL: https://issues.apache.org/jira/browse/FLINK-6830
> Project: Flink
>  Issue Type: Test
>  Components: State Backends, Checkpointing
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
> Fix For: 1.3.1
>
>
> Already with FLINK-6763 and FLINK-6764 we'll need to change the serialization 
> formats between 1.3.0 and 1.3.x.
> We probably should add the stateful job migration ITCases for restoring from 
> Flink 1.3.x now.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-6830) Add ITTests for savepoint migration from 1.3

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038436#comment-16038436
 ] 

ASF GitHub Bot commented on FLINK-6830:
---

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/4059#discussion_r120303440
  
--- Diff: 
flink-tests/src/test/java/org/apache/flink/test/state/operator/restore/unkeyed/ChainBreakTest.java
 ---
@@ -31,8 +36,22 @@
 /**
  * Verifies that the state of all operators is restored if a topology 
change breaks up a chain.
  */
+@RunWith(Parameterized.class)
 public class ChainBreakTest extends 
AbstractNonKeyedOperatorRestoreTestBase {
 
+   private final String savepointPath;
+
+   @Parameterized.Parameters(name = "Migrate Savepoint: {0}")
+   public static Collection parameters () {
--- End diff --

is it not possible to move this into the abstract super class?


> Add ITTests for savepoint migration from 1.3
> 
>
> Key: FLINK-6830
> URL: https://issues.apache.org/jira/browse/FLINK-6830
> Project: Flink
>  Issue Type: Test
>  Components: State Backends, Checkpointing
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
> Fix For: 1.3.1
>
>
> Already with FLINK-6763 and FLINK-6764 we'll need to change the serialization 
> formats between 1.3.0 and 1.3.x.
> We probably should add the stateful job migration ITCases for restoring from 
> Flink 1.3.x now.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-6830) Add ITTests for savepoint migration from 1.3

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16036119#comment-16036119
 ] 

ASF GitHub Bot commented on FLINK-6830:
---

Github user tzulitai commented on the issue:

https://github.com/apache/flink/pull/4059
  
f228081 is for porting the topology change on restore migration tests.

The rest 233c617 to e619b98 is for ports of each individual migration test 
with the pattern `*From12MigrationTest`. They're also refactored to deduplicate 
code across 1.1, 1.2, 1.3 migration tests.

Note that for the CEP migration tests in 233c617, some tests for migration 
from 1.3 is currently ignored because they do not pass. May have bumped into a 
bug for the CEP library that does not allow it to restore from 1.3.x to 1.3.x 


> Add ITTests for savepoint migration from 1.3
> 
>
> Key: FLINK-6830
> URL: https://issues.apache.org/jira/browse/FLINK-6830
> Project: Flink
>  Issue Type: Test
>  Components: State Backends, Checkpointing
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
> Fix For: 1.3.1
>
>
> Already with FLINK-6763 and FLINK-6764 we'll need to change the serialization 
> formats between 1.3.0 and 1.3.x.
> We probably should add the stateful job migration ITCases for restoring from 
> Flink 1.3.x now.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-6830) Add ITTests for savepoint migration from 1.3

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16036046#comment-16036046
 ] 

ASF GitHub Bot commented on FLINK-6830:
---

Github user tzulitai commented on the issue:

https://github.com/apache/flink/pull/4059
  
@aljoscha +1 to that, I'm actually working on those also already :-D


> Add ITTests for savepoint migration from 1.3
> 
>
> Key: FLINK-6830
> URL: https://issues.apache.org/jira/browse/FLINK-6830
> Project: Flink
>  Issue Type: Test
>  Components: State Backends, Checkpointing
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
> Fix For: 1.3.1
>
>
> Already with FLINK-6763 and FLINK-6764 we'll need to change the serialization 
> formats between 1.3.0 and 1.3.x.
> We probably should add the stateful job migration ITCases for restoring from 
> Flink 1.3.x now.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-6830) Add ITTests for savepoint migration from 1.3

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16036045#comment-16036045
 ] 

ASF GitHub Bot commented on FLINK-6830:
---

Github user aljoscha commented on the issue:

https://github.com/apache/flink/pull/4059
  
@tzulitai While you're on it, what would you think about also porting the 
`*From12MigrationTest` tests?


> Add ITTests for savepoint migration from 1.3
> 
>
> Key: FLINK-6830
> URL: https://issues.apache.org/jira/browse/FLINK-6830
> Project: Flink
>  Issue Type: Test
>  Components: State Backends, Checkpointing
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
> Fix For: 1.3.1
>
>
> Already with FLINK-6763 and FLINK-6764 we'll need to change the serialization 
> formats between 1.3.0 and 1.3.x.
> We probably should add the stateful job migration ITCases for restoring from 
> Flink 1.3.x now.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-6830) Add ITTests for savepoint migration from 1.3

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16035889#comment-16035889
 ] 

ASF GitHub Bot commented on FLINK-6830:
---

Github user tzulitai commented on the issue:

https://github.com/apache/flink/pull/4059
  
Thanks for the reviews @zentol @aljoscha! I'll add the topology change 
ITTests also and then merge this.


> Add ITTests for savepoint migration from 1.3
> 
>
> Key: FLINK-6830
> URL: https://issues.apache.org/jira/browse/FLINK-6830
> Project: Flink
>  Issue Type: Test
>  Components: State Backends, Checkpointing
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
> Fix For: 1.3.1
>
>
> Already with FLINK-6763 and FLINK-6764 we'll need to change the serialization 
> formats between 1.3.0 and 1.3.x.
> We probably should add the stateful job migration ITCases for restoring from 
> Flink 1.3.x now.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-6830) Add ITTests for savepoint migration from 1.3

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16035874#comment-16035874
 ] 

ASF GitHub Bot commented on FLINK-6830:
---

Github user aljoscha commented on the issue:

https://github.com/apache/flink/pull/4059
  
The changes look good! @zentol's suggestion also seems wise.  


> Add ITTests for savepoint migration from 1.3
> 
>
> Key: FLINK-6830
> URL: https://issues.apache.org/jira/browse/FLINK-6830
> Project: Flink
>  Issue Type: Test
>  Components: State Backends, Checkpointing
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
> Fix For: 1.3.1
>
>
> Already with FLINK-6763 and FLINK-6764 we'll need to change the serialization 
> formats between 1.3.0 and 1.3.x.
> We probably should add the stateful job migration ITCases for restoring from 
> Flink 1.3.x now.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-6830) Add ITTests for savepoint migration from 1.3

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16034877#comment-16034877
 ] 

ASF GitHub Bot commented on FLINK-6830:
---

Github user zentol commented on the issue:

https://github.com/apache/flink/pull/4059
  
Can we also include the tests for the restore with topology changes in 
flink-tests?

org.apache.flink.test.start.operator.restore *


> Add ITTests for savepoint migration from 1.3
> 
>
> Key: FLINK-6830
> URL: https://issues.apache.org/jira/browse/FLINK-6830
> Project: Flink
>  Issue Type: Test
>  Components: State Backends, Checkpointing
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
> Fix For: 1.3.1
>
>
> Already with FLINK-6763 and FLINK-6764 we'll need to change the serialization 
> formats between 1.3.0 and 1.3.x.
> We probably should add the stateful job migration ITCases for restoring from 
> Flink 1.3.x now.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-6830) Add ITTests for savepoint migration from 1.3

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16034858#comment-16034858
 ] 

ASF GitHub Bot commented on FLINK-6830:
---

GitHub user tzulitai opened a pull request:

https://github.com/apache/flink/pull/4059

[FLINK-6830] Add StatefulJobSavepointFrom13MigrationITCase

With [FLINK-6763](https://issues.apache.org/jira/browse/FLINK-6763) and 
[FLINK-6764](https://issues.apache.org/jira/browse/FLINK-6764) we'll need to 
change the serialization formats between 1.3.0 and 1.3.x.

This PR adds the ITCase for the savepoint migration so that those format 
changes will be properly guarded for backwards compatibility. The savepoint 
binary files of this PR were taken using the current `release-1.3` branch.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tzulitai/flink FLINK-6830

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4059.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4059


commit fb2b1ddbe52b160a8c45052ca6abd9c0f5477d7d
Author: Tzu-Li (Gordon) Tai 
Date:   2017-06-02T15:18:51Z

[FLINK-6830] Add StatefulJobSavepointFrom13MigrationITCase




> Add ITTests for savepoint migration from 1.3
> 
>
> Key: FLINK-6830
> URL: https://issues.apache.org/jira/browse/FLINK-6830
> Project: Flink
>  Issue Type: Test
>  Components: State Backends, Checkpointing
>Affects Versions: 1.3.0
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
> Fix For: 1.3.1
>
>
> Already with FLINK-6763 and FLINK-6764 we'll need to change the serialization 
> formats between 1.3.0 and 1.3.x.
> We probably should add the stateful job migration ITCases for restoring from 
> Flink 1.3.x now.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)