chenboat commented on a change in pull request #3877: Pinot controller side
change to enhance LLC segment metadata upload.
URL: https://github.com/apache/incubator-pinot/pull/3877#discussion_r260556395
##########
File path:
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManager.java
##########
@@ -544,8 +547,18 @@ protected boolean updateOldSegmentMetadataZNRecord(String
realtimeTableName, LLC
String rawTableName =
TableNameBuilder.extractRawTableName(realtimeTableName);
committingSegmentMetadata.setDownloadUrl(
ControllerConf.constructDownloadUrl(rawTableName,
committingSegmentNameStr, _controllerConf.generateVipUrl()));
- // Pull segment metadata from incoming segment and set it in zk segment
metadata
- SegmentMetadataImpl segmentMetadata = extractSegmentMetadata(rawTableName,
committingSegmentNameStr);
+ // Pull segment metadata from committingSegmentDescriptor if present.
Otherwise extract it from the incoming segment
+ // and set it in zk segment metadata.
+ // Note: ideally we should use the interface SegmentMetadata here for the
class of segmentMetadata. However,
+ // SegmentMetadataImpl has member like ColumnMetadata which is not in the
interface. Adding these members to the
+ // interface will introduce circular dependency between Pinot-Core and
Pinot-common.
+ SegmentMetadataImpl segmentMetadata;
+ if (committingSegmentDescriptor.getSegmentMetadata() != null &&
+ committingSegmentDescriptor.getSegmentMetadata() instanceof
SegmentMetadataImpl) {
+ segmentMetadata =
(SegmentMetadataImpl)committingSegmentDescriptor.getSegmentMetadata();
+ } else {
+ segmentMetadata = extractSegmentMetadata(rawTableName,
committingSegmentNameStr);
Review comment:
Move segment metadata extraction to the LLCSegmentCompletionHandler.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
