[
https://issues.apache.org/jira/browse/FLINK-36216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17879221#comment-17879221
]
Robert Metzger edited comment on FLINK-36216 at 9/4/24 12:48 PM:
-----------------------------------------------------------------
Some initial research on the issue:
{code}
at GlobalWindowAggsHandler$473.merge_split229(Unknown Source)
at GlobalWindowAggsHandler$473.merge(Unknown Source)
{code}
--> Seems to be generated code from the Table API
This issue seems related: https://github.com/facebook/rocksdb/issues/8248
Questions in that ticket:
bq. Were the input files created by a pre-6.14 RocksDB version,
Flink 1.19 uses 6.20.3-ververica-2.0, so NO.
bq. or without check_flush_compaction_key_order,
bq. or with a different comparator?
I couldn't find any info on that from the default rocksdb logs. I
[suspect|https://rocksdb.org/blog/2021/05/26/online-validation.html] those are
ColumnFamilyOptions, which are not printed by default.
was (Author: rmetzger):
Some initial research on the issue:
{code}
at GlobalWindowAggsHandler$473.merge_split229(Unknown Source)
at GlobalWindowAggsHandler$473.merge(Unknown Source)
{code}
--> Seems to be generated code from the Table API
This issue seems related: https://github.com/facebook/rocksdb/issues/8248
Questions in that ticket:
> Were the input files created by a pre-6.14 RocksDB version,
Flink 1.19 uses 6.20.3-ververica-2.0, so NO.
> or without check_flush_compaction_key_order,
> or with a different comparator?
I couldn't find any info on that from the default rocksdb logs. I
[suspect|https://rocksdb.org/blog/2021/05/26/online-validation.html] those are
ColumnFamilyOptions, which are not printed by default.
> RocksDB: Compaction sees out-of-order keys
> ------------------------------------------
>
> Key: FLINK-36216
> URL: https://issues.apache.org/jira/browse/FLINK-36216
> Project: Flink
> Issue Type: Bug
> Components: Runtime / State Backends
> Affects Versions: 1.19.0
> Reporter: Robert Metzger
> Priority: Major
>
> {code}
> org.rocksdb.RocksDBException: Compaction sees out-of-order keys.
> at org.rocksdb.RocksDB.put(Native Method)
> at org.rocksdb.RocksDB.put(RocksDB.java:955)
> at
> org.apache.flink.contrib.streaming.state.RocksDBMapState.put(RocksDBMapState.java:139)
> at
> org.apache.flink.table.runtime.dataview.StateMapView$StateMapViewWithKeysNullable.put(StateMapView.java:168)
> at
> org.apache.flink.table.runtime.dataview.StateMapView$NamespacedStateMapViewWithKeysNullable.put(StateMapView.java:392)
> at GlobalWindowAggsHandler$473.merge_split229(Unknown Source)
> at GlobalWindowAggsHandler$473.merge(Unknown Source)
> at
> org.apache.flink.table.runtime.operators.aggregate.window.combines.GlobalAggCombiner.combineAccumulator(GlobalAggCombiner.java:99)
> at
> org.apache.flink.table.runtime.operators.aggregate.window.combines.GlobalAggCombiner.combine(GlobalAggCombiner.java:85)
> at
> org.apache.flink.table.runtime.operators.aggregate.window.buffers.RecordsWindowBuffer.flush(RecordsWindowBuffer.java:112)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
