VenuReddy2103 commented on a change in pull request #3819:
URL: https://github.com/apache/carbondata/pull/3819#discussion_r479973113
##########
File path:
sdk/sdk/src/main/java/org/apache/carbondata/sdk/file/CarbonWriterBuilder.java
##########
@@ -660,13 +895,39 @@ public CarbonWriter build() throws IOException,
InvalidLoadOptionException {
// removed from the load. LoadWithoutConverter flag is going to point to
the Loader Builder
// which will skip Conversion Step.
loadModel.setLoadWithoutConverterStep(true);
- return new AvroCarbonWriter(loadModel, hadoopConf, this.avroSchema);
+ AvroCarbonWriter avroCarbonWriter = new AvroCarbonWriter(loadModel,
Review comment:
We have some code duplications for each type of writer. Suggest to
refactor it. Something like this -
```suggestion
CarbonWriter carbonWriter;
if (this.writerType == WRITER_TYPE.AVRO) {
// AVRO records are pushed to Carbon as Object not as Strings. This
was done in order to
// handle multi level complex type support. As there are no conversion
converter step is
// removed from the load. LoadWithoutConverter flag is going to point
to the Loader Builder
// which will skip Conversion Step.
loadModel.setLoadWithoutConverterStep(true);
carbonWriter = new AvroCarbonWriter(loadModel, hadoopConf,
this.avroSchema);
} else if (this.writerType == WRITER_TYPE.JSON) {
loadModel.setJsonFileLoad(true);
carbonWriter = new JsonCarbonWriter(loadModel, hadoopConf);
} else if (this.writerType == WRITER_TYPE.PARQUET) {
loadModel.setLoadWithoutConverterStep(true);
carbonWriter = new ParquetCarbonWriter(loadModel, hadoopConf,
this.avroSchema);
} else if (this.writerType == WRITER_TYPE.ORC) {
carbonWriter = new ORCCarbonWriter(loadModel, hadoopConf);
} else {
// CSV
CSVCarbonWriter csvCarbonWriter = new CSVCarbonWriter(loadModel,
hadoopConf);
if (!this.options.containsKey(CarbonCommonConstants.FILE_HEADER)) {
csvCarbonWriter.setSkipHeader(true);
}
carbonWriter = csvCarbonWriter;
}
if (!StringUtils.isEmpty(filePath)) {
carbonWriter.validateAndSetDataFiles(this.dataFiles);
}
return carbonWriter;
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
