mcvsubbu commented on a change in pull request #3877: Pinot controller side
change to enhance LLC segment metadata upload.
URL: https://github.com/apache/incubator-pinot/pull/3877#discussion_r264890953
##########
File path:
pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/LLCSegmentCompletionHandlers.java
##########
@@ -182,33 +195,46 @@ public String
segmentCommitStart(@QueryParam(SegmentCompletionProtocol.PARAM_INS
@Path(SegmentCompletionProtocol.MSG_TYPE_COMMIT_END)
@Produces(MediaType.APPLICATION_JSON)
public String
segmentCommitEnd(@QueryParam(SegmentCompletionProtocol.PARAM_INSTANCE_ID)
String instanceId,
- @QueryParam(SegmentCompletionProtocol.PARAM_SEGMENT_NAME) String
segmentName,
- @QueryParam(SegmentCompletionProtocol.PARAM_SEGMENT_LOCATION) String
segmentLocation,
- @QueryParam(SegmentCompletionProtocol.PARAM_OFFSET) long offset,
- @QueryParam(SegmentCompletionProtocol.PARAM_MEMORY_USED_BYTES) long
memoryUsedBytes,
- @QueryParam(SegmentCompletionProtocol.PARAM_BUILD_TIME_MILLIS) long
buildTimeMillis,
- @QueryParam(SegmentCompletionProtocol.PARAM_WAIT_TIME_MILLIS) long
waitTimeMillis,
- @QueryParam(SegmentCompletionProtocol.PARAM_ROW_COUNT) int numRows,
- @QueryParam(SegmentCompletionProtocol.PARAM_SEGMENT_SIZE_BYTES) long
segmentSizeBytes) {
+
@QueryParam(SegmentCompletionProtocol.PARAM_SEGMENT_NAME) String segmentName,
+
@QueryParam(SegmentCompletionProtocol.PARAM_SEGMENT_LOCATION) String
segmentLocation,
+
@QueryParam(SegmentCompletionProtocol.PARAM_OFFSET) long offset,
+
@QueryParam(SegmentCompletionProtocol.PARAM_MEMORY_USED_BYTES) long
memoryUsedBytes,
+
@QueryParam(SegmentCompletionProtocol.PARAM_BUILD_TIME_MILLIS) long
buildTimeMillis,
+
@QueryParam(SegmentCompletionProtocol.PARAM_WAIT_TIME_MILLIS) long
waitTimeMillis,
+
@QueryParam(SegmentCompletionProtocol.PARAM_ROW_COUNT) int numRows,
+
@QueryParam(SegmentCompletionProtocol.PARAM_SEGMENT_SIZE_BYTES) long
segmentSizeBytes) {
if (instanceId == null || segmentName == null || offset == -1 ||
segmentLocation == null) {
LOGGER.error("Invalid call: offset={}, segmentName={}, instanceId={},
segmentLocation={}", offset, segmentName,
- instanceId, segmentLocation);
+ instanceId, segmentLocation);
// TODO: memoryUsedInBytes = 0 if not present in params. Add validation
when we start using it
return SegmentCompletionProtocol.RESP_FAILED.toJsonString();
}
+ SegmentMetadataImpl segmentMetadata;
+ try {
+ segmentMetadata = extractMetadataFromSegmentFile(segmentName, new
URI(segmentLocation));
+ } catch (URISyntaxException e) {
+ LOGGER.error("Invalid segment location: ", segmentLocation);
Review comment:
Please include segment name in this log. The idea is that we should be able
to grep a segment name in the logs and find out what happened to it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
