JingsongLi commented on code in PR #8104:
URL: https://github.com/apache/paimon/pull/8104#discussion_r3348902075
##########
paimon-python/pypaimon/common/options/core_options.py:
##########
@@ -885,6 +900,12 @@ def data_file_external_paths_strategy(self, default=None):
def data_file_external_paths_specific_fs(self, default=None):
return
self.options.get(CoreOptions.DATA_FILE_EXTERNAL_PATHS_SPECIFIC_FS, default)
+ def data_file_external_paths_weights(self, default=None):
+ value = self.options.get(CoreOptions.DATA_FILE_EXTERNAL_PATHS_WEIGHTS,
default)
+ if not value:
+ return None
+ return [int(w.strip()) for w in value.split(",") if w.strip()]
Review Comment:
Please validate the parsed weights here, matching Java
`CoreOptions.externalPathWeights()`. Currently zero or negative weights are
accepted and then `WeightedExternalPathProvider` builds invalid cumulative
weights: for example `0,0` makes `total_weight` zero so every random value maps
past the last cumulative entry and all files go to the last path, while `10,-5`
makes the second path unreachable. Java rejects non-positive weights, so Python
should raise for `<= 0` too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
