nsivabalan commented on a change in pull request #2923:
URL: https://github.com/apache/hudi/pull/2923#discussion_r636207743
##########
File path: hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java
##########
@@ -485,6 +488,10 @@ private static Object
convertValueForAvroLogicalTypes(Schema fieldSchema, Object
return decimalConversion.fromBytes((ByteBuffer) fieldValue,
fieldSchema,
LogicalTypes.decimal(dc.getPrecision(), dc.getScale()));
}
+ } else if (fieldSchema.getLogicalType() == LogicalTypes.timestampMicros())
{
+ return Instant.EPOCH.plus(Long.parseLong(fieldValue.toString()),
ChronoUnit.MICROS).atZone(ZoneId.systemDefault()).toLocalDateTime();
+ } else if (fieldSchema.getLogicalType() == LogicalTypes.timestampMillis())
{
Review comment:
can you please point me to testcase which is covering this.
##########
File path:
hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/keygen/TestTimestampBasedKeyGenerator.java
##########
@@ -378,4 +382,46 @@ public void
test_ExpectsMatch_MultipleInputFormats_ShortDate_OutputCustomDate()
baseRow = genericRecordToRow(baseRecord);
assertEquals("04/01/2020", keyGen.getPartitionPath(baseRow));
}
+
+ @Test
+ public void test_ExpectsMatch_LogicalType_Date() throws IOException {
+ LocalDate today = LocalDate.now();
+ baseRecord.put("dob", (int) today.toEpochDay());
+ properties = this.getBaseKeyConfig(
+ "DATE",
+ "yyyy-MM-dd'T'HH:mm:ssZ,yyyy-MM-dd'T'HH:mm:ss.SSSZ,yyyyMMdd",
+ "",
+ null,
+ "yyyy-MM-dd",
+ DateTimeZone.getDefault().getID());
+
+ properties.setProperty(KeyGeneratorOptions.PARTITIONPATH_FIELD_OPT_KEY,
"dob");
+ BuiltinKeyGenerator keyGen = new TimestampBasedKeyGenerator(properties);
+ HoodieKey hk1 = keyGen.getKey(baseRecord);
+ Assertions.assertEquals(today.toString(), hk1.getPartitionPath());
+
+ baseRow = genericRecordToRow(baseRecord);
+ assertEquals(today.toString(), keyGen.getPartitionPath(baseRow));
+ }
+
+ @Test
+ public void test_ExpectsMatch_LogicalType_Timestamp() throws IOException {
Review comment:
I see we have added LocalDate, LocalDatetime, Date and Timestamp support
to source code, but tests are added only for 2 of these. can we please add for
rest as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
