abhishekagarwal87 commented on code in PR #15090:
URL: https://github.com/apache/druid/pull/15090#discussion_r1350003044
##########
processing/src/main/java/org/apache/druid/timeline/partition/OvershadowableManager.java:
##########
@@ -418,9 +420,11 @@ private Iterator<Entry<RootPartitionRange,
Short2ObjectSortedMap<AtomicUpdateGro
TreeMap<RootPartitionRange, Short2ObjectSortedMap<AtomicUpdateGroup<T>>>
stateMap
)
{
- final RootPartitionRange lowFench = new RootPartitionRange(partitionId,
partitionId);
+ // remediate submap `fromKey > toKey` issue when partitionId overflows
+ final short partitionIdLowFence = partitionId < 0 ? Short.MAX_VALUE :
partitionId;
Review Comment:
@dulu98Kurz - I am not entirely sure but `short` was likely chosen to save
on memory that storing these ids will take. 32K partitions in one single
interval is too high. Can you describe a bit more as to how your cluster ends
up in this situation and why is that a genuine scenario? In my experience,
almost every time, an interval touching this high number means that compaction
is not configured or ingestion is misconfigured.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
