yanghua commented on a change in pull request #2340:
URL: https://github.com/apache/hudi/pull/2340#discussion_r544373942
##########
File path:
hudi-spark-datasource/hudi-spark/src/main/java/org/apache/hudi/QuickstartUtils.java
##########
@@ -121,11 +121,22 @@ public static GenericRecord generateGenericRecord(String
rowKey, String riderNam
*/
public static OverwriteWithLatestAvroPayload generateRandomValue(HoodieKey
key, String riderDriverSuffix)
throws IOException {
+ // The timestamp generated is limited to range from 7 days before to
now, to avoid generating too many
+ // partitionPaths when user use timestamp as partitionPath filed.
GenericRecord rec =
- generateGenericRecord(key.getRecordKey(), "rider-" +
riderDriverSuffix, "driver-" + riderDriverSuffix, 0);
+ generateGenericRecord(key.getRecordKey(), "rider-" +
riderDriverSuffix,"driver-"
Review comment:
Miss whitespace before `"driver-"`
##########
File path:
hudi-spark-datasource/hudi-spark/src/main/java/org/apache/hudi/QuickstartUtils.java
##########
@@ -121,11 +121,22 @@ public static GenericRecord generateGenericRecord(String
rowKey, String riderNam
*/
public static OverwriteWithLatestAvroPayload generateRandomValue(HoodieKey
key, String riderDriverSuffix)
throws IOException {
+ // The timestamp generated is limited to range from 7 days before to
now, to avoid generating too many
+ // partitionPaths when user use timestamp as partitionPath filed.
GenericRecord rec =
- generateGenericRecord(key.getRecordKey(), "rider-" +
riderDriverSuffix, "driver-" + riderDriverSuffix, 0);
+ generateGenericRecord(key.getRecordKey(), "rider-" +
riderDriverSuffix,"driver-"
+ + riderDriverSuffix, generateRangeRandomTimestamp(7));
return new OverwriteWithLatestAvroPayload(Option.of(rec));
}
+ /**
+ * Generate timestamp range from {@param daysTillNow} before to now.
+ */
+ private static long generateRangeRandomTimestamp(int daysTillNow) {
+ long interval = daysTillNow * 24 * 60 * 60 * 1000L;
Review comment:
Does `maxIntervalMillis` look better?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
