[jira] [Commented] (FLINK-4000) Exception: Could not restore checkpointed state to operators and functions; during Job Restart (Job restart is triggered due to one of the task manager failure)

2016-06-08 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/FLINK-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320527#comment-15320527
 ] 

ASF GitHub Bot commented on FLINK-4000:
---

Github user asfgit closed the pull request at:

https://github.com/apache/flink/pull/2062


> Exception: Could not restore checkpointed state to operators and functions;  
> during Job Restart (Job restart is triggered due to one of the task manager 
> failure)
> -
>
> Key: FLINK-4000
> URL: https://issues.apache.org/jira/browse/FLINK-4000
> Project: Flink
>  Issue Type: Bug
>  Components: DataStream API
>Affects Versions: 1.0.3
> Environment: //Fault Tolerance Configuration of the Job
> env.enableCheckpointing(5000); 
> env.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);
> env.getCheckpointConfig().setMaxConcurrentCheckpoints(1);
> env.setRestartStrategy(RestartStrategies.fixedDelayRestart( 3,1));
>Reporter: Aride Chettali
>
> java.lang.Exception: Could not restore checkpointed state to operators and 
> functions
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:457)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:209)
>   at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.Exception: Failed to restore state to function: null
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:168)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:449)
>   ... 3 more
> Caused by: java.lang.NullPointerException
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:184)
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:80)
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:165)
>   ... 4 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (FLINK-4000) Exception: Could not restore checkpointed state to operators and functions; during Job Restart (Job restart is triggered due to one of the task manager failure)

2016-06-07 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/FLINK-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318783#comment-15318783
 ] 

ASF GitHub Bot commented on FLINK-4000:
---

Github user StephanEwen commented on the issue:

https://github.com/apache/flink/pull/2062
  
Thanks, this is a critical fix.
+1

Merging this...


> Exception: Could not restore checkpointed state to operators and functions;  
> during Job Restart (Job restart is triggered due to one of the task manager 
> failure)
> -
>
> Key: FLINK-4000
> URL: https://issues.apache.org/jira/browse/FLINK-4000
> Project: Flink
>  Issue Type: Bug
>  Components: DataStream API
>Affects Versions: 1.0.3
> Environment: //Fault Tolerance Configuration of the Job
> env.enableCheckpointing(5000); 
> env.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);
> env.getCheckpointConfig().setMaxConcurrentCheckpoints(1);
> env.setRestartStrategy(RestartStrategies.fixedDelayRestart( 3,1));
>Reporter: Aride Chettali
>
> java.lang.Exception: Could not restore checkpointed state to operators and 
> functions
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:457)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:209)
>   at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.Exception: Failed to restore state to function: null
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:168)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:449)
>   ... 3 more
> Caused by: java.lang.NullPointerException
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:184)
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:80)
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:165)
>   ... 4 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (FLINK-4000) Exception: Could not restore checkpointed state to operators and functions; during Job Restart (Job restart is triggered due to one of the task manager failure)

2016-06-01 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/FLINK-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311404#comment-15311404
 ] 

ASF GitHub Bot commented on FLINK-4000:
---

Github user rekhajoshm closed the pull request at:

https://github.com/apache/flink/pull/2061


> Exception: Could not restore checkpointed state to operators and functions;  
> during Job Restart (Job restart is triggered due to one of the task manager 
> failure)
> -
>
> Key: FLINK-4000
> URL: https://issues.apache.org/jira/browse/FLINK-4000
> Project: Flink
>  Issue Type: Bug
>  Components: DataStream API
>Affects Versions: 1.0.3
> Environment: //Fault Tolerance Configuration of the Job
> env.enableCheckpointing(5000); 
> env.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);
> env.getCheckpointConfig().setMaxConcurrentCheckpoints(1);
> env.setRestartStrategy(RestartStrategies.fixedDelayRestart( 3,1));
>Reporter: Aride Chettali
>Assignee: Rekha Joshi
>
> java.lang.Exception: Could not restore checkpointed state to operators and 
> functions
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:457)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:209)
>   at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.Exception: Failed to restore state to function: null
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:168)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:449)
>   ... 3 more
> Caused by: java.lang.NullPointerException
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:184)
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:80)
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:165)
>   ... 4 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (FLINK-4000) Exception: Could not restore checkpointed state to operators and functions; during Job Restart (Job restart is triggered due to one of the task manager failure)

2016-06-01 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/FLINK-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311403#comment-15311403
 ] 

ASF GitHub Bot commented on FLINK-4000:
---

Github user rekhajoshm commented on the issue:

https://github.com/apache/flink/pull/2061
  
aha, in one of my intermediate runs had done just initializing 
idsProcessedButNotAcknowledged and  retain pendingCheckpoints , but in last run 
changed it calling open() :-( . makes sense @asavartsov. closing. thanks


> Exception: Could not restore checkpointed state to operators and functions;  
> during Job Restart (Job restart is triggered due to one of the task manager 
> failure)
> -
>
> Key: FLINK-4000
> URL: https://issues.apache.org/jira/browse/FLINK-4000
> Project: Flink
>  Issue Type: Bug
>  Components: DataStream API
>Affects Versions: 1.0.3
> Environment: //Fault Tolerance Configuration of the Job
> env.enableCheckpointing(5000); 
> env.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);
> env.getCheckpointConfig().setMaxConcurrentCheckpoints(1);
> env.setRestartStrategy(RestartStrategies.fixedDelayRestart( 3,1));
>Reporter: Aride Chettali
>Assignee: Rekha Joshi
>
> java.lang.Exception: Could not restore checkpointed state to operators and 
> functions
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:457)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:209)
>   at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.Exception: Failed to restore state to function: null
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:168)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:449)
>   ... 3 more
> Caused by: java.lang.NullPointerException
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:184)
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:80)
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:165)
>   ... 4 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (FLINK-4000) Exception: Could not restore checkpointed state to operators and functions; during Job Restart (Job restart is triggered due to one of the task manager failure)

2016-06-01 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/FLINK-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311395#comment-15311395
 ] 

ASF GitHub Bot commented on FLINK-4000:
---

Github user asavartsov commented on the issue:

https://github.com/apache/flink/pull/2061
  
Take a look at my pull request at https://github.com/apache/flink/pull/2062


> Exception: Could not restore checkpointed state to operators and functions;  
> during Job Restart (Job restart is triggered due to one of the task manager 
> failure)
> -
>
> Key: FLINK-4000
> URL: https://issues.apache.org/jira/browse/FLINK-4000
> Project: Flink
>  Issue Type: Bug
>  Components: DataStream API
>Affects Versions: 1.0.3
> Environment: //Fault Tolerance Configuration of the Job
> env.enableCheckpointing(5000); 
> env.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);
> env.getCheckpointConfig().setMaxConcurrentCheckpoints(1);
> env.setRestartStrategy(RestartStrategies.fixedDelayRestart( 3,1));
>Reporter: Aride Chettali
>Assignee: Rekha Joshi
>
> java.lang.Exception: Could not restore checkpointed state to operators and 
> functions
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:457)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:209)
>   at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.Exception: Failed to restore state to function: null
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:168)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:449)
>   ... 3 more
> Caused by: java.lang.NullPointerException
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:184)
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:80)
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:165)
>   ... 4 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (FLINK-4000) Exception: Could not restore checkpointed state to operators and functions; during Job Restart (Job restart is triggered due to one of the task manager failure)

2016-06-01 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/FLINK-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311375#comment-15311375
 ] 

ASF GitHub Bot commented on FLINK-4000:
---

Github user rekhajoshm commented on the issue:

https://github.com/apache/flink/pull/2061
  
@asavartsov Ok. Please let me know how you propose to solve this? thanks


> Exception: Could not restore checkpointed state to operators and functions;  
> during Job Restart (Job restart is triggered due to one of the task manager 
> failure)
> -
>
> Key: FLINK-4000
> URL: https://issues.apache.org/jira/browse/FLINK-4000
> Project: Flink
>  Issue Type: Bug
>  Components: DataStream API
>Affects Versions: 1.0.3
> Environment: //Fault Tolerance Configuration of the Job
> env.enableCheckpointing(5000); 
> env.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);
> env.getCheckpointConfig().setMaxConcurrentCheckpoints(1);
> env.setRestartStrategy(RestartStrategies.fixedDelayRestart( 3,1));
>Reporter: Aride Chettali
>Assignee: Rekha Joshi
>
> java.lang.Exception: Could not restore checkpointed state to operators and 
> functions
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:457)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:209)
>   at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.Exception: Failed to restore state to function: null
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:168)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:449)
>   ... 3 more
> Caused by: java.lang.NullPointerException
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:184)
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:80)
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:165)
>   ... 4 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (FLINK-4000) Exception: Could not restore checkpointed state to operators and functions; during Job Restart (Job restart is triggered due to one of the task manager failure)

2016-06-01 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/FLINK-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311362#comment-15311362
 ] 

ASF GitHub Bot commented on FLINK-4000:
---

Github user asavartsov commented on the issue:

https://github.com/apache/flink/pull/2061
  
No, it does not make any sense and even makes things worse, sorry.




> Exception: Could not restore checkpointed state to operators and functions;  
> during Job Restart (Job restart is triggered due to one of the task manager 
> failure)
> -
>
> Key: FLINK-4000
> URL: https://issues.apache.org/jira/browse/FLINK-4000
> Project: Flink
>  Issue Type: Bug
>  Components: DataStream API
>Affects Versions: 1.0.3
> Environment: //Fault Tolerance Configuration of the Job
> env.enableCheckpointing(5000); 
> env.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);
> env.getCheckpointConfig().setMaxConcurrentCheckpoints(1);
> env.setRestartStrategy(RestartStrategies.fixedDelayRestart( 3,1));
>Reporter: Aride Chettali
>Assignee: Rekha Joshi
>
> java.lang.Exception: Could not restore checkpointed state to operators and 
> functions
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:457)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:209)
>   at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.Exception: Failed to restore state to function: null
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:168)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:449)
>   ... 3 more
> Caused by: java.lang.NullPointerException
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:184)
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:80)
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:165)
>   ... 4 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (FLINK-4000) Exception: Could not restore checkpointed state to operators and functions; during Job Restart (Job restart is triggered due to one of the task manager failure)

2016-06-01 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/FLINK-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311356#comment-15311356
 ] 

ASF GitHub Bot commented on FLINK-4000:
---

GitHub user asavartsov opened a pull request:

https://github.com/apache/flink/pull/2062

[FLINK-4000] Fix for checkpoint state restore at 
MessageAcknowledgingSourceBase

As says documentation for MessageAcknowledgingSourceBase.restoreState() 

This method is invoked when a function is executed as part of a recovery 
run. Note that restoreState() is called before open().

So current implementation

1. Fails on restoreState with NullPointerException, jobs fail to restart.
2. Does not restore anything because following open erases all checkpoint 
data immediately.
3. As consequence, violates exactly once rule because processed but not 
acknowledged list erased.

Proposed change fixes that.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/asavartsov/flink FLINK-4000

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/2062.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2062


commit 58fb7df0b2d1dd68a90f4088fbde10874eb290b6
Author: Alexey Savartsov 
Date:   2016-06-01T23:23:53Z

[FLINK-4000] Fix for checkpoint state restore at 
MessageAcknowledgingSourceBase

As says documentation for MessageAcknowledgingSourceBase.restoreState() 

This method is invoked when a function is executed as part of a recovery 
run. Note that restoreState() is called before open().

So current implementation

1. Fails on restoreState with NullPointerException, jobs fail to restart.
2. Does not restore anything because following open erases all checkpoint 
data immediately.
3. As consequence, violates exactly once rule because processed but not 
acknowledged list erased.

Proposed change fixes that.




> Exception: Could not restore checkpointed state to operators and functions;  
> during Job Restart (Job restart is triggered due to one of the task manager 
> failure)
> -
>
> Key: FLINK-4000
> URL: https://issues.apache.org/jira/browse/FLINK-4000
> Project: Flink
>  Issue Type: Bug
>  Components: DataStream API
>Affects Versions: 1.0.3
> Environment: //Fault Tolerance Configuration of the Job
> env.enableCheckpointing(5000); 
> env.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);
> env.getCheckpointConfig().setMaxConcurrentCheckpoints(1);
> env.setRestartStrategy(RestartStrategies.fixedDelayRestart( 3,1));
>Reporter: Aride Chettali
>Assignee: Rekha Joshi
>
> java.lang.Exception: Could not restore checkpointed state to operators and 
> functions
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:457)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:209)
>   at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.Exception: Failed to restore state to function: null
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:168)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:449)
>   ... 3 more
> Caused by: java.lang.NullPointerException
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:184)
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:80)
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:165)
>   ... 4 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (FLINK-4000) Exception: Could not restore checkpointed state to operators and functions; during Job Restart (Job restart is triggered due to one of the task manager failure)

2016-06-01 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/FLINK-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311206#comment-15311206
 ] 

ASF GitHub Bot commented on FLINK-4000:
---

Github user rekhajoshm commented on the issue:

https://github.com/apache/flink/pull/2061
  
I agree @asavartsov , that was a quick look and was working to 
reproduce.Does the updated make sense? thank you.


> Exception: Could not restore checkpointed state to operators and functions;  
> during Job Restart (Job restart is triggered due to one of the task manager 
> failure)
> -
>
> Key: FLINK-4000
> URL: https://issues.apache.org/jira/browse/FLINK-4000
> Project: Flink
>  Issue Type: Bug
>  Components: DataStream API
>Affects Versions: 1.0.3
> Environment: //Fault Tolerance Configuration of the Job
> env.enableCheckpointing(5000); 
> env.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);
> env.getCheckpointConfig().setMaxConcurrentCheckpoints(1);
> env.setRestartStrategy(RestartStrategies.fixedDelayRestart( 3,1));
>Reporter: Aride Chettali
>Assignee: Rekha Joshi
>
> java.lang.Exception: Could not restore checkpointed state to operators and 
> functions
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:457)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:209)
>   at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.Exception: Failed to restore state to function: null
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:168)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:449)
>   ... 3 more
> Caused by: java.lang.NullPointerException
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:184)
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:80)
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:165)
>   ... 4 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (FLINK-4000) Exception: Could not restore checkpointed state to operators and functions; during Job Restart (Job restart is triggered due to one of the task manager failure)

2016-06-01 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/FLINK-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311152#comment-15311152
 ] 

ASF GitHub Bot commented on FLINK-4000:
---

Github user asavartsov commented on the issue:

https://github.com/apache/flink/pull/2061
  
This kind of check is might be useless and probably wouldn't fix the issue. 
My debugging shows that the list `idsProcessedButNotAcknowledged` is null on 
recovery, not checkpoints itself. This list is initialized in `open` method, 
but somehow it doesn't get called in such scenario.


> Exception: Could not restore checkpointed state to operators and functions;  
> during Job Restart (Job restart is triggered due to one of the task manager 
> failure)
> -
>
> Key: FLINK-4000
> URL: https://issues.apache.org/jira/browse/FLINK-4000
> Project: Flink
>  Issue Type: Bug
>  Components: DataStream API
>Affects Versions: 1.0.3
> Environment: //Fault Tolerance Configuration of the Job
> env.enableCheckpointing(5000); 
> env.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);
> env.getCheckpointConfig().setMaxConcurrentCheckpoints(1);
> env.setRestartStrategy(RestartStrategies.fixedDelayRestart( 3,1));
>Reporter: Aride Chettali
>Assignee: Rekha Joshi
>
> java.lang.Exception: Could not restore checkpointed state to operators and 
> functions
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:457)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:209)
>   at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.Exception: Failed to restore state to function: null
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:168)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:449)
>   ... 3 more
> Caused by: java.lang.NullPointerException
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:184)
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:80)
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:165)
>   ... 4 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (FLINK-4000) Exception: Could not restore checkpointed state to operators and functions; during Job Restart (Job restart is triggered due to one of the task manager failure)

2016-06-01 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/FLINK-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311087#comment-15311087
 ] 

ASF GitHub Bot commented on FLINK-4000:
---

GitHub user rekhajoshm opened a pull request:

https://github.com/apache/flink/pull/2061

[FLINK-4000] Checkpoint dictionaries null after taskmgr failures

Fix for exception during job restart after task mgr failure, at which point 
restoreState fails as checkpoint dictionaries can be null.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/rekhajoshm/flink FLINK-4000

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/2061.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2061


commit 1e1ce9efefccf5a5585be802af4200e9d1ae7a98
Author: Rekha Joshi 
Date:   2016-06-01T20:03:50Z

Merge pull request #1 from apache/master

Apache Flink master pull

commit 88929dcd989c34c3232f29a96af1b32ec1315911
Author: Joshi 
Date:   2016-06-01T20:56:52Z

[FLINK-4000] Checkpoint dictionaries null after taskmgr failures




> Exception: Could not restore checkpointed state to operators and functions;  
> during Job Restart (Job restart is triggered due to one of the task manager 
> failure)
> -
>
> Key: FLINK-4000
> URL: https://issues.apache.org/jira/browse/FLINK-4000
> Project: Flink
>  Issue Type: Bug
>  Components: DataStream API
>Affects Versions: 1.0.3
> Environment: //Fault Tolerance Configuration of the Job
> env.enableCheckpointing(5000); 
> env.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);
> env.getCheckpointConfig().setMaxConcurrentCheckpoints(1);
> env.setRestartStrategy(RestartStrategies.fixedDelayRestart( 3,1));
>Reporter: Aride Chettali
>Assignee: Rekha Joshi
>
> java.lang.Exception: Could not restore checkpointed state to operators and 
> functions
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:457)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:209)
>   at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.Exception: Failed to restore state to function: null
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:168)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:449)
>   ... 3 more
> Caused by: java.lang.NullPointerException
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:184)
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:80)
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:165)
>   ... 4 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (FLINK-4000) Exception: Could not restore checkpointed state to operators and functions; during Job Restart (Job restart is triggered due to one of the task manager failure)

2016-06-01 Thread Alexey Savartsov (JIRA)

[ 
https://issues.apache.org/jira/browse/FLINK-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310328#comment-15310328
 ] 

Alexey Savartsov commented on FLINK-4000:
-

I can confirm this issue with RMQSource, with zookeeper state too. Looks like 
restoreState on MessageAcknowledgingSourceBase is called before open, so 
dictionaries for checkpoints are null.

> Exception: Could not restore checkpointed state to operators and functions;  
> during Job Restart (Job restart is triggered due to one of the task manager 
> failure)
> -
>
> Key: FLINK-4000
> URL: https://issues.apache.org/jira/browse/FLINK-4000
> Project: Flink
>  Issue Type: Bug
>  Components: DataStream API
>Affects Versions: 1.0.3
> Environment: //Fault Tolerance Configuration of the Job
> env.enableCheckpointing(5000); 
> env.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);
> env.getCheckpointConfig().setMaxConcurrentCheckpoints(1);
> env.setRestartStrategy(RestartStrategies.fixedDelayRestart( 3,1));
>Reporter: Aride Chettali
>
> java.lang.Exception: Could not restore checkpointed state to operators and 
> functions
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:457)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:209)
>   at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.Exception: Failed to restore state to function: null
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:168)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:449)
>   ... 3 more
> Caused by: java.lang.NullPointerException
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:184)
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:80)
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:165)
>   ... 4 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (FLINK-4000) Exception: Could not restore checkpointed state to operators and functions; during Job Restart (Job restart is triggered due to one of the task manager failure)

2016-06-01 Thread Aride Chettali (JIRA)

[ 
https://issues.apache.org/jira/browse/FLINK-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310323#comment-15310323
 ] 

Aride Chettali commented on FLINK-4000:
---

Thank you for the quick response.
Yes I am using the default state checkpoint (in-memory). 

Looks like the error is in source function,I am using Rabbit MQ source with 
correlationIds enabled. Below is the dependency I have
 
org.apache.flink
flink-connector-rabbitmq_2.10
1.0.3
 


> Exception: Could not restore checkpointed state to operators and functions;  
> during Job Restart (Job restart is triggered due to one of the task manager 
> failure)
> -
>
> Key: FLINK-4000
> URL: https://issues.apache.org/jira/browse/FLINK-4000
> Project: Flink
>  Issue Type: Bug
>  Components: DataStream API
>Affects Versions: 1.0.3
> Environment: //Fault Tolerance Configuration of the Job
> env.enableCheckpointing(5000); 
> env.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);
> env.getCheckpointConfig().setMaxConcurrentCheckpoints(1);
> env.setRestartStrategy(RestartStrategies.fixedDelayRestart( 3,1));
>Reporter: Aride Chettali
>
> java.lang.Exception: Could not restore checkpointed state to operators and 
> functions
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:457)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:209)
>   at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.Exception: Failed to restore state to function: null
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:168)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:449)
>   ... 3 more
> Caused by: java.lang.NullPointerException
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:184)
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:80)
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:165)
>   ... 4 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (FLINK-4000) Exception: Could not restore checkpointed state to operators and functions; during Job Restart (Job restart is triggered due to one of the task manager failure)

2016-06-01 Thread Aride Chettali (JIRA)

[ 
https://issues.apache.org/jira/browse/FLINK-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310251#comment-15310251
 ] 

Aride Chettali commented on FLINK-4000:
---

I was using the default state check-points (in-memory). 
And I have figured out the issue. The issue was user function did not implement 
the "Checkpointed" interface. Once I implemented the methods snapshotState() 
and restoreState() then my job got restarted successfully.
Thanks for your help..

> Exception: Could not restore checkpointed state to operators and functions;  
> during Job Restart (Job restart is triggered due to one of the task manager 
> failure)
> -
>
> Key: FLINK-4000
> URL: https://issues.apache.org/jira/browse/FLINK-4000
> Project: Flink
>  Issue Type: Bug
>  Components: DataStream API
>Affects Versions: 1.0.3
> Environment: //Fault Tolerance Configuration of the Job
> env.enableCheckpointing(5000); 
> env.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);
> env.getCheckpointConfig().setMaxConcurrentCheckpoints(1);
> env.setRestartStrategy(RestartStrategies.fixedDelayRestart( 3,1));
>Reporter: Aride Chettali
>
> java.lang.Exception: Could not restore checkpointed state to operators and 
> functions
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:457)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:209)
>   at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.Exception: Failed to restore state to function: null
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:168)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:449)
>   ... 3 more
> Caused by: java.lang.NullPointerException
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:184)
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:80)
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:165)
>   ... 4 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (FLINK-4000) Exception: Could not restore checkpointed state to operators and functions; during Job Restart (Job restart is triggered due to one of the task manager failure)

2016-06-01 Thread Aride Chettali (JIRA)

[ 
https://issues.apache.org/jira/browse/FLINK-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310250#comment-15310250
 ] 

Aride Chettali commented on FLINK-4000:
---

I was using the default state check-points (in-memory).  
And I have figured out the issue. The issue was user function did not implement 
the "Checkpointed" interface. Once I implemented the functions _snapshotState_  
and _restoreState_ then my job got restarted successfully.
 
Thanks for your help..

> Exception: Could not restore checkpointed state to operators and functions;  
> during Job Restart (Job restart is triggered due to one of the task manager 
> failure)
> -
>
> Key: FLINK-4000
> URL: https://issues.apache.org/jira/browse/FLINK-4000
> Project: Flink
>  Issue Type: Bug
>  Components: DataStream API
>Affects Versions: 1.0.3
> Environment: //Fault Tolerance Configuration of the Job
> env.enableCheckpointing(5000); 
> env.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);
> env.getCheckpointConfig().setMaxConcurrentCheckpoints(1);
> env.setRestartStrategy(RestartStrategies.fixedDelayRestart( 3,1));
>Reporter: Aride Chettali
>
> java.lang.Exception: Could not restore checkpointed state to operators and 
> functions
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:457)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:209)
>   at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.Exception: Failed to restore state to function: null
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:168)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:449)
>   ... 3 more
> Caused by: java.lang.NullPointerException
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:184)
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:80)
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:165)
>   ... 4 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (FLINK-4000) Exception: Could not restore checkpointed state to operators and functions; during Job Restart (Job restart is triggered due to one of the task manager failure)

2016-06-01 Thread Ufuk Celebi (JIRA)

[ 
https://issues.apache.org/jira/browse/FLINK-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310007#comment-15310007
 ] 

Ufuk Celebi commented on FLINK-4000:


Thanks for reporting the issue. Did you configure a specific state backend or 
are you using the default one?

> Exception: Could not restore checkpointed state to operators and functions;  
> during Job Restart (Job restart is triggered due to one of the task manager 
> failure)
> -
>
> Key: FLINK-4000
> URL: https://issues.apache.org/jira/browse/FLINK-4000
> Project: Flink
>  Issue Type: Bug
>  Components: DataStream API
>Affects Versions: 1.0.3
> Environment: //Fault Tolerance Configuration of the Job
> env.enableCheckpointing(5000); 
> env.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);
> env.getCheckpointConfig().setMaxConcurrentCheckpoints(1);
> env.setRestartStrategy(RestartStrategies.fixedDelayRestart( 3,1));
>Reporter: Aride Chettali
>
> java.lang.Exception: Could not restore checkpointed state to operators and 
> functions
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:457)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:209)
>   at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.Exception: Failed to restore state to function: null
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:168)
>   at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreState(StreamTask.java:449)
>   ... 3 more
> Caused by: java.lang.NullPointerException
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:184)
>   at 
> org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase.restoreState(MessageAcknowledgingSourceBase.java:80)
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.restoreState(AbstractUdfStreamOperator.java:165)
>   ... 4 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)