mcvsubbu commented on a change in pull request #5639:
URL: https://github.com/apache/incubator-pinot/pull/5639#discussion_r448660319
##########
File path:
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManager.java
##########
@@ -355,10 +355,17 @@ void setIdealState(String realtimeTableName, IdealState
idealState) {
* This method moves the segment file from another location to its permanent
location.
* When splitCommit is enabled, segment file is uploaded to the
segmentLocation in the committingSegmentDescriptor,
* and we need to move the segment file to its permanent location before
committing the segment metadata.
+ * Updated the segment location to the uri which the segment is moved to.
+ * Exception: for committingSegmentDescriptor with peer download scheme in
its segment location, there is no need for
+ * moving.
Review comment:
delete this line
##########
File path:
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManager.java
##########
@@ -355,10 +355,17 @@ void setIdealState(String realtimeTableName, IdealState
idealState) {
* This method moves the segment file from another location to its permanent
location.
* When splitCommit is enabled, segment file is uploaded to the
segmentLocation in the committingSegmentDescriptor,
* and we need to move the segment file to its permanent location before
committing the segment metadata.
+ * Updated the segment location to the uri which the segment is moved to.
Review comment:
```suggestion
* Modifies the segment location in committingSegmentDescriptor to the uri
which the segment is moved to
```
##########
File path:
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManager.java
##########
@@ -355,10 +355,17 @@ void setIdealState(String realtimeTableName, IdealState
idealState) {
* This method moves the segment file from another location to its permanent
location.
* When splitCommit is enabled, segment file is uploaded to the
segmentLocation in the committingSegmentDescriptor,
* and we need to move the segment file to its permanent location before
committing the segment metadata.
+ * Updated the segment location to the uri which the segment is moved to.
+ * Exception: for committingSegmentDescriptor with peer download scheme in
its segment location, there is no need for
Review comment:
```suggestion
* unless committingSegmentDescriptor has a peer download uri scheme in
segment location.
```
##########
File path:
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManager.java
##########
@@ -485,9 +498,10 @@ private LLCRealtimeSegmentZKMetadata
updateCommittingSegmentZKMetadata(String re
// TODO Issue 5953 remove the long parsing once metadata is set correctly.
committingSegmentZKMetadata.setEndOffset(committingSegmentDescriptor.getNextOffset());
committingSegmentZKMetadata.setStatus(Status.DONE);
- committingSegmentZKMetadata.setDownloadUrl(URIUtils
- .constructDownloadUrl(_controllerConf.generateVipUrl(),
TableNameBuilder.extractRawTableName(realtimeTableName),
- segmentName));
+ // If the download url set by the server is a peer download url format
with peer scheme, put an empty string in zk
+ // otherwise just use the location in the descriptor.
+
committingSegmentZKMetadata.setDownloadUrl(isPeerURL(committingSegmentDescriptor.getSegmentLocation())
? ""
Review comment:
Can we define "" as `public static string
METADATA_URI_FOR_PEER_DOWNLOAD` in `CommonConstants.Segment` and reference the
same thing in the server as well, to detect peer download? Once we decide it is
going to be `""` all components need to use the same thing, so might as well
make a common definition. A change to this will need to be done keeping
backward compat in mind, if at all someone wants to change it.
thanks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
