mcvsubbu commented on a change in pull request #4713: Refactoring realtime
segment committer
URL: https://github.com/apache/incubator-pinot/pull/4713#discussion_r335695269
##########
File path:
pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/LLRealtimeSegmentDataManager.java
##########
@@ -837,22 +779,24 @@ protected boolean
commitSegment(SegmentCompletionProtocol.Response response) {
return true;
}
- protected SegmentCompletionProtocol.Response postSegmentCommitMsg() {
- final File segmentTarFile = new
File(_segmentBuildDescriptor.getSegmentTarFilePath());
+ protected SegmentCompletionProtocol.Response
commit(SegmentCompletionProtocol.Response response, boolean isSplitCommit) {
SegmentCompletionProtocol.Request.Params params = new
SegmentCompletionProtocol.Request.Params();
-
params.withInstanceId(_instanceId).withOffset(_currentOffset).withSegmentName(_segmentNameStr)
- .withNumRows(_numRowsConsumed).withInstanceId(_instanceId)
- .withBuildTimeMillis(_segmentBuildDescriptor.getBuildTimeMillis())
+
+
params.withSegmentName(_segmentNameStr).withOffset(_currentOffset).withNumRows(_numRowsConsumed)
+
.withInstanceId(_instanceId).withBuildTimeMillis(_segmentBuildDescriptor.getBuildTimeMillis())
.withSegmentSizeBytes(_segmentBuildDescriptor.getSegmentSizeBytes())
.withWaitTimeMillis(_segmentBuildDescriptor.getWaitTimeMillis());
if (_isOffHeap) {
params.withMemoryUsedBytes(_memoryManager.getTotalAllocatedBytes());
}
- SegmentCompletionProtocol.Response response =
_protocolHandler.segmentCommit(params, segmentTarFile);
- if
(!response.getStatus().equals(SegmentCompletionProtocol.ControllerResponseStatus.COMMIT_SUCCESS))
{
- segmentLogger.warn("Commit failed with response {}",
response.toJsonString());
+
+ if (isSplitCommit) {
+ SplitSegmentCommitter splitSegmentCommitter = new
SplitSegmentCommitter(params, _protocolHandler, response,
_indexLoadingConfig.isEnableSplitCommitEndWithMetadata(), segmentLogger);
Review comment:
Just pass the _indexLoadingConfig.
Use a factory to create the committer class.
You can construct the factory up front, with constriuctor(segmentLogger,
_protocolHandler, and indexLoadingConfig)
A createCommitter() method on the factory that can take (params, response)
as arguments and create the right kind of committer.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
