jenniferdai commented on a change in pull request #4244: Add support for custom
record-readers in the create-segment tool
URL: https://github.com/apache/incubator-pinot/pull/4244#discussion_r289531085
##########
File path:
pinot-tools/src/main/java/org/apache/pinot/tools/admin/command/CreateSegmentCommand.java
##########
@@ -228,15 +228,17 @@ public boolean execute()
}
}
- FileFormat configFormat = segmentGeneratorConfig.getFormat();
- if (_format == null) {
- if (configFormat == null) {
- throw new RuntimeException("Format cannot be null in config file.");
- }
- _format = configFormat;
- } else {
- if (configFormat != _format && configFormat != FileFormat.AVRO) {
- LOGGER.warn("Find format conflict in command line and config file, use
config in command line: {}", _format);
+ if (segmentGeneratorConfig.getRecordReaderPath() == null) {
Review comment:
// Allow for instantiation general record readers from a record reader
path passed into segment generator config
// If this is set, this will override the file format
if (recordReaderPath != null) {
if (fileFormat != FileFormat.OTHER) {
// NOTE: we currently have default file format set to AVRO inside
segment generator config, do not want to break
// this behavior for clients.
LOGGER
.warn("Using class: {} to read segment, ignoring configured file
format: {}", recordReaderPath, fileFormat);
}
Currently, in record reader factory, we have this default. Up to you, but
potentially, we could follow the same model though requiring users to set file
format to other seems annoying... maybe we can just leave a comment that the
behavior is different or put it in relevant docs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
