dulu98Kurz commented on code in PR #15090:
URL: https://github.com/apache/druid/pull/15090#discussion_r1350608551
##########
processing/src/main/java/org/apache/druid/timeline/partition/OvershadowableManager.java:
##########
@@ -418,9 +420,11 @@ private Iterator<Entry<RootPartitionRange,
Short2ObjectSortedMap<AtomicUpdateGro
TreeMap<RootPartitionRange, Short2ObjectSortedMap<AtomicUpdateGroup<T>>>
stateMap
)
{
- final RootPartitionRange lowFench = new RootPartitionRange(partitionId,
partitionId);
+ // remediate submap `fromKey > toKey` issue when partitionId overflows
+ final short partitionIdLowFence = partitionId < 0 ? Short.MAX_VALUE :
partitionId;
Review Comment:
So instead of capping at `short` max, we can possibly cover the gap by:
- Allowing partitionId goes into `Int` range
- Logging error messages to strongly remind user we need to compact/reduce
num of segments.
For users who do not have late messages or compaction issues, this change
has no impact to them because they won't store more than `short` max segments
anyway, so we don`t break the initial intension of saving on mem.
For users who actually can produce segments beyond `short` max, this will
buy them more time to compact/reduc number of segments, which may eventually
avoid the difficult situation above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
