mcvsubbu commented on a change in pull request #4914: [POC] By-passing
deep-store requirement for Realtime segment completion
URL: https://github.com/apache/incubator-pinot/pull/4914#discussion_r360458689
##########
File path:
pinot-common/src/main/java/org/apache/pinot/common/protocols/SegmentCompletionProtocol.java
##########
@@ -186,7 +189,8 @@ public String getUrl(String hostPort, String protocol) {
+ (_params.getSegmentSizeBytes() <= 0 ? ""
: ("&" + PARAM_SEGMENT_SIZE_BYTES + "=" +
_params.getSegmentSizeBytes())) + (_params.getNumRows() <= 0 ? ""
: ("&" + PARAM_ROW_COUNT + "=" + _params.getNumRows())) +
(_params.getSegmentLocation() == null ? ""
- : ("&" + PARAM_SEGMENT_LOCATION + "=" +
_params.getSegmentLocation()));
+ : ("&" + PARAM_SEGMENT_LOCATION + "=" +
_params.getSegmentLocation()))
+ + "&" + PARAM_SEGMENT_UPLOAD_TO_CONTROLLER + "=" +
_params.getSegmentUploadToController();
Review comment:
I dont quite understand what this parameter means. I guess it is used in the
3rd part of the split commit? Does it mean that the server has uploaded the
segment to deep store? Btw, a recent (already merged) PR has changed the
semantics of split commit, so that it should _always_ include metadata in
end-commit. Split commit without segment metadata in phase-3 is now deprecated,
and we should not develop features on it. I am not sure how that applies here,
but just stating to make it clear.
I suggest that the segment location parameter contain a special value if the
upload to deep store was not successful, but peer download is available. Say,
"peer://uri/of/segment".
Alternatively, a boolean indication saying the segment was NOT successfully
uploaded to deep store.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
