jihoonson commented on a change in pull request #10288:
URL: https://github.com/apache/druid/pull/10288#discussion_r492438779
##########
File path:
core/src/main/java/org/apache/druid/timeline/partition/BuildingHashBasedNumberedShardSpec.java
##########
@@ -58,6 +61,7 @@ public BuildingHashBasedNumberedShardSpec(
this.partitionDimensions = partitionDimensions == null
?
HashBasedNumberedShardSpec.DEFAULT_PARTITION_DIMENSIONS
: partitionDimensions;
+ this.partitionFunction = partitionFunction;
Review comment:
Good point. I thought it will break some things at first.. But, thinking
about it more, I think having a default in partitionsSpec will be fine even
though there are at least 2 issues.
- In auto compaction, the last compaction state of each segment includes the
partitionsSpec of the compaction task which created the segment. The auto
compaction searches for all segments which don't have the last compaction state
matched to the current configuration. So, for the segments compacted before,
the `HashedPartitionsSpec` can have an empty partition function. This can be
translated to different default hash functions if we ever change the default
hash function. Then the auto compaction can silently ignore the changed hash
function. However, this doesn't seem that bad.
- In rolling upgrades which replace machines of an old version with ones of
a new version, there can be a mixed version of middleManagers or indexers. In
this case, all parallel tasks will be broken as each subtask can use different
partition functions if we have changed the default partition function between
those two versions. I think this will be acceptable as long as we announce it
as a known issue in the release notes. (And we are not probably going to change
the default partition function in the near future.)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
