[
https://issues.apache.org/jira/browse/BEAM-7126?focusedWorklogId=240808&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-240808
]
ASF GitHub Bot logged work on BEAM-7126:
----------------------------------------
Author: ASF GitHub Bot
Created on: 13/May/19 02:01
Start Date: 13/May/19 02:01
Worklog Time Spent: 10m
Work Description: tweise commented on issue #8472: [BEAM-7126] Remove
duplicate key encoding for state backend
URL: https://github.com/apache/beam/pull/8472#issuecomment-491652141
This yields the same error that I saw earlier (during checkpointing):
```
AsynchronousException{java.lang.Exception: Could not materialize checkpoint
1 for operator [1]statefulCount (83/128).}
at
org.apache.flink.streaming.runtime.tasks.StreamTask$AsyncCheckpointExceptionHandler.tryHandleCheckpointException(StreamTask.java:1154)
at
org.apache.flink.streaming.runtime.tasks.StreamTask$AsyncCheckpointRunnable.handleExecutionException(StreamTask.java:948)
at
org.apache.flink.streaming.runtime.tasks.StreamTask$AsyncCheckpointRunnable.run(StreamTask.java:885)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.Exception: Could not materialize checkpoint 1 for
operator [1]statefulCount (83/128).
at
org.apache.flink.streaming.runtime.tasks.StreamTask$AsyncCheckpointRunnable.handleExecutionException(StreamTask.java:943)
... 6 more
Caused by: java.util.concurrent.ExecutionException:
java.lang.ArrayIndexOutOfBoundsException: 88
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at
org.apache.flink.util.FutureUtil.runIfNotDoneAndGet(FutureUtil.java:53)
at
org.apache.flink.streaming.api.operators.OperatorSnapshotFinalizer.<init>(OperatorSnapshotFinalizer.java:47)
at
org.apache.flink.streaming.runtime.tasks.StreamTask$AsyncCheckpointRunnable.run(StreamTask.java:854)
... 5 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: 88
at
org.apache.flink.runtime.state.heap.CopyOnWriteStateTableSnapshot.partitionEntriesByKeyGroup(CopyOnWriteStateTableSnapshot.java:147)
at
org.apache.flink.runtime.state.heap.CopyOnWriteStateTableSnapshot.writeMappingsInKeyGroup(CopyOnWriteStateTableSnapshot.java:178)
at
org.apache.flink.runtime.state.heap.HeapKeyedStateBackend$HeapSnapshotStrategy$1.performOperation(HeapKeyedStateBackend.java:697)
at
org.apache.flink.runtime.state.heap.HeapKeyedStateBackend$HeapSnapshotStrategy$1.performOperation(HeapKeyedStateBackend.java:641)
at
org.apache.flink.runtime.io.async.AbstractAsyncCallableWithResources.call(AbstractAsyncCallableWithResources.java:75)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
org.apache.flink.util.FutureUtil.runIfNotDoneAndGet(FutureUtil.java:50)
... 7 more
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 240808)
Time Spent: 40m (was: 0.5h)
> Double encoding of state keys in portable Flink runner
> ------------------------------------------------------
>
> Key: BEAM-7126
> URL: https://issues.apache.org/jira/browse/BEAM-7126
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
> Affects Versions: 2.11.0
> Reporter: Thomas Weise
> Assignee: Maximilian Michels
> Priority: Major
> Labels: portability-flink
> Time Spent: 40m
> Remaining Estimate: 0h
>
> State keys currently need to be encoded as NESTED. My attempt to use the
> ByteString directly in BEAM-7112 caused checkpointing to fail. We should look
> into eliminating the redundant key encoding and adjusting
> StateRequestHandlers.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
