jackylk commented on a change in pull request #3606: [CARBONDATA-3681] Change
default compressor to zstd
URL: https://github.com/apache/carbondata/pull/3606#discussion_r377100880
##########
File path:
core/src/main/java/org/apache/carbondata/core/util/path/CarbonTablePath.java
##########
@@ -285,17 +286,38 @@ public static String getSegmentPath(String tablePath,
String segmentId) {
}
/**
- * Gets data file name only with out path
- *
- * @param filePartNo data file part number
- * @param taskNo task identifier
- * @param factUpdateTimeStamp unique identifier to identify an update
- * @return gets data file name only with out path
+ * Gets data file name only, without parent path
*/
public static String getCarbonDataFileName(Integer filePartNo, String
taskNo, int bucketNumber,
- int batchNo, String factUpdateTimeStamp, String segmentNo) {
- return DATA_PART_PREFIX + filePartNo + "-" + taskNo + BATCH_PREFIX +
batchNo + "-"
- + bucketNumber + "-" + segmentNo + "-" + factUpdateTimeStamp +
CARBON_DATA_EXT;
+ int batchNo, String factUpdateTimeStamp, String segmentNo, String
compressor) {
+ Objects.requireNonNull(filePartNo);
+ Objects.requireNonNull(taskNo);
+ Objects.requireNonNull(factUpdateTimeStamp);
+ Objects.requireNonNull(compressor);
+
+ // Start from CarbonData 2.0, the data file name patten is:
+ // partNo-taskNo-batchNo-bucketNo-segmentNo-timestamp.compressor.carbondata
+ // For example:
+ // part-0-0_batchno0-0-0-1580982686749.zstd.carbondata
+ //
+ // If the compressor name is missing, the file is compressed by snappy,
which is
+ // the default compressor in CarbonData 1.x
+
+ return new StringBuilder()
+ .append(DATA_PART_PREFIX)
+ .append(filePartNo)
+ .append("-")
+ .append(taskNo)
+ .append(BATCH_PREFIX)
+ .append(batchNo)
+ .append("-")
+ .append(bucketNumber)
+ .append("-")
+ .append(segmentNo)
+ .append("-")
+ .append(factUpdateTimeStamp)
Review comment:
yes, I wanted to test once without changing the file name, in order to
simulate using new jar to test on old files.
As the CI is passed, so that the compatibility is ok.
I will add the compressor name in file name now
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
