[
https://issues.apache.org/jira/browse/FLINK-34581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824224#comment-17824224
]
Henning Schmiedehausen commented on FLINK-34581:
------------------------------------------------
I am using the MiniClusterExtension as part of a unit test. Does that load the
flink-conf.yaml? Where would it need to be located?
I don't use an actual cluster or deploy to a cluster (yet). Would I need the
same open statement for running locally as well?
> streaming code throws java.lang.reflect.InaccessibleObjectException
> -------------------------------------------------------------------
>
> Key: FLINK-34581
> URL: https://issues.apache.org/jira/browse/FLINK-34581
> Project: Flink
> Issue Type: Bug
> Components: API / DataStream
> Affects Versions: 1.18.1
> Reporter: Henning Schmiedehausen
> Priority: Blocker
>
> I have a pretty simple test pipeline (read a bunch of tables from Apache
> Kafka, join and project them, then write to Apache Iceberg) that I run
> locally with Junit5 and the MiniClusterExtension. This works ok with Java 11
> and Flink 1.17.2 and Flink 1.18.1.
> With Java 17, I see
> {{[WARN ] IcebergStreamWriter (1/1)#1
> (78d9251dbbab3aae84bf303dfc080d23_626c1e687bcaad0c13c507629a5894a8_0_1)
> switched from RUNNING to FAILED with failure cause:}}
> {{java.io.IOException: Could not perform checkpoint 2 for operator
> IcebergStreamWriter (1/1)#1.}}
> {{at
> org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointOnBarrier(StreamTask.java:1275)}}
> {{at
> org.apache.flink.streaming.runtime.io.checkpointing.CheckpointBarrierHandler.notifyCheckpoint(CheckpointBarrierHandler.java:147)}}
> {{at
> org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.triggerCheckpoint(SingleCheckpointBarrierHandler.java:287)}}
> {{at
> org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.access$100(SingleCheckpointBarrierHandler.java:64)}}
> {{at
> org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler$ControllerImpl.triggerGlobalCheckpoint(SingleCheckpointBarrierHandler.java:488)}}
> {{at
> org.apache.flink.streaming.runtime.io.checkpointing.AbstractAlignedBarrierHandlerState.triggerGlobalCheckpoint(AbstractAlignedBarrierHandlerState.java:74)}}
> {{at
> org.apache.flink.streaming.runtime.io.checkpointing.AbstractAlignedBarrierHandlerState.barrierReceived(AbstractAlignedBarrierHandlerState.java:66)}}
> {{at
> org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.lambda$processBarrier$2(SingleCheckpointBarrierHandler.java:234)}}
> {{at
> org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.markCheckpointAlignedAndTransformState(SingleCheckpointBarrierHandler.java:262)}}
> {{at
> org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.processBarrier(SingleCheckpointBarrierHandler.java:231)}}
> {{at
> org.apache.flink.streaming.runtime.io.checkpointing.CheckpointedInputGate.handleEvent(CheckpointedInputGate.java:181)}}
> {{at
> org.apache.flink.streaming.runtime.io.checkpointing.CheckpointedInputGate.pollNext(CheckpointedInputGate.java:159)}}
> {{at
> org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:118)}}
> {{at
> org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)}}
> {{at
> org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:562)}}
> {{at
> org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231)}}
> {{at
> org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:858)}}
> {{at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:807)}}
> {{at
> org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:953)}}
> {{at
> org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:932)}}
> {{at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:746)}}
> {{at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562)}}
> {{at java.base/java.lang.Thread.run(Thread.java:840)}}
> {{Caused by: java.lang.RuntimeException:
> java.lang.reflect.InaccessibleObjectException: Unable to make field private
> final java.lang.Object[] java.util.Arrays$ArrayList.a accessible: module
> java.base does not "opens java.util" to unnamed module @6af93788}}
> {{at
> com.twitter.chill.java.ArraysAsListSerializer.<init>(ArraysAsListSerializer.java:69)}}
> {{at
> org.apache.flink.api.java.typeutils.runtime.kryo.FlinkChillPackageRegistrar.registerSerializers(FlinkChillPackageRegistrar.java:67)}}
> {{at
> org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.getKryoInstance(KryoSerializer.java:513)}}
> {{at
> org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.checkKryoInitialized(KryoSerializer.java:522)}}
> {{at
> org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.serialize(KryoSerializer.java:348)}}
> {{at
> org.apache.flink.streaming.runtime.streamrecord.StreamElementSerializer.serialize(StreamElementSerializer.java:165)}}
> {{at
> org.apache.flink.streaming.runtime.streamrecord.StreamElementSerializer.serialize(StreamElementSerializer.java:43)}}
> {{at
> org.apache.flink.runtime.plugable.SerializationDelegate.write(SerializationDelegate.java:54)}}
> {{at
> org.apache.flink.runtime.io.network.api.writer.RecordWriter.serializeRecord(RecordWriter.java:141)}}
> {{at
> org.apache.flink.runtime.io.network.api.writer.RecordWriter.emit(RecordWriter.java:107)}}
> {{at
> org.apache.flink.runtime.io.network.api.writer.ChannelSelectorRecordWriter.emit(ChannelSelectorRecordWriter.java:55)}}
> {{at
> org.apache.flink.streaming.runtime.io.RecordWriterOutput.pushToRecordWriter(RecordWriterOutput.java:134)}}
> {{at
> org.apache.flink.streaming.runtime.io.RecordWriterOutput.collectAndCheckIfChained(RecordWriterOutput.java:114)}}
> {{at
> org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:95)}}
> {{at
> org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:48)}}
> {{at
> org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:59)}}
> {{at
> org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:31)}}
> {{at
> org.apache.iceberg.flink.sink.IcebergStreamWriter.flush(IcebergStreamWriter.java:113)}}
> {{at
> org.apache.iceberg.flink.sink.IcebergStreamWriter.prepareSnapshotPreBarrier(IcebergStreamWriter.java:66)}}
> {{at
> org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.prepareSnapshotPreBarrier(RegularOperatorChain.java:89)}}
> {{at
> org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.checkpointState(SubtaskCheckpointCoordinatorImpl.java:322)}}
> {{at
> org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$performCheckpoint$15(StreamTask.java:1318)}}
> {{at
> org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50)}}
> {{at
> org.apache.flink.streaming.runtime.tasks.StreamTask.performCheckpoint(StreamTask.java:1306)}}
> {{at
> org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointOnBarrier(StreamTask.java:1263)}}
> {{... 22 more}}
> {{Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make
> field private final java.lang.Object[] java.util.Arrays$ArrayList.a
> accessible: module java.base does not "opens java.util" to unnamed module
> @6af93788}}
> {{at
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)}}
> {{at
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)}}
> {{at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:178)}}
> {{at java.base/java.lang.reflect.Field.setAccessible(Field.java:172)}}
> {{at
> com.twitter.chill.java.ArraysAsListSerializer.<init>(ArraysAsListSerializer.java:67)}}
> {{... 46 more}}
> This is blocking us right now from upgrading to Java 17.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
