npawar commented on a change in pull request #5597:
URL: https://github.com/apache/incubator-pinot/pull/5597#discussion_r443704676
##########
File path:
pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/HLRealtimeSegmentDataManager.java
##########
@@ -169,9 +169,9 @@ public HLRealtimeSegmentDataManager(final
RealtimeSegmentZKMetadata realtimeSegm
// create and init stream level consumer
StreamConsumerFactory streamConsumerFactory =
StreamConsumerFactoryProvider.create(_streamConfig);
String clientId = HLRealtimeSegmentDataManager.class.getSimpleName() + "-"
+ _streamConfig.getTopicName();
- _streamLevelConsumer = streamConsumerFactory
- .createStreamLevelConsumer(clientId, _tableNameWithType,
SchemaUtils.extractSourceFields(schema),
- instanceMetadata.getGroupId(_tableNameWithType));
+ _streamLevelConsumer =
streamConsumerFactory.createStreamLevelConsumer(clientId, _tableNameWithType,
+ IngestionUtils.getFieldsForRecordExtractor(tableConfig, schema),
Review comment:
1. Yes the explanation is right. Updated javadoc on
IngestionUtils#getFieldsForRecordExtractor method.
2. Individual calls should never have to be made, I feel it is better if
provide only the method which will extract all the necessary fields, so that we
avoid errors in the following phases of ingestion.
3. I like the suggestion of passing just IngestionConfig to set the
discipline. Changed the method to take IngestionConfig.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
