vinothchandar commented on a change in pull request #1541:
URL: https://github.com/apache/incubator-hudi/pull/1541#discussion_r412685370
##########
File path:
hudi-utilities/src/main/java/org/apache/hudi/utilities/keygen/TimestampBasedKeyGenerator.java
##########
@@ -123,4 +132,8 @@ public HoodieKey getKey(GenericRecord record) {
throw new HoodieDeltaStreamerException("Unable to parse input partition
field :" + partitionVal, pe);
}
}
+
+ private long convertLongTimeToSeconds(Long partitionVal) {
Review comment:
this is converting to ms, rather?
##########
File path:
hudi-utilities/src/main/java/org/apache/hudi/utilities/keygen/TimestampBasedKeyGenerator.java
##########
@@ -35,16 +35,21 @@
import java.util.Collections;
import java.util.Date;
import java.util.TimeZone;
+import java.util.concurrent.TimeUnit;
+
+import static java.util.concurrent.TimeUnit.MILLISECONDS;
/**
* Key generator, that relies on timestamps for partitioning field. Still
picks record key by name.
*/
public class TimestampBasedKeyGenerator extends SimpleKeyGenerator {
enum TimestampType implements Serializable {
- UNIX_TIMESTAMP, DATE_STRING, MIXED, EPOCHMILLISECONDS
Review comment:
this will break users setting EPOCHMILLISECONDS today.. Could we avoid
this pain and transparently handle this
##########
File path:
hudi-utilities/src/main/java/org/apache/hudi/utilities/keygen/TimestampBasedKeyGenerator.java
##########
@@ -35,16 +35,21 @@
import java.util.Collections;
import java.util.Date;
import java.util.TimeZone;
+import java.util.concurrent.TimeUnit;
+
+import static java.util.concurrent.TimeUnit.MILLISECONDS;
/**
* Key generator, that relies on timestamps for partitioning field. Still
picks record key by name.
*/
public class TimestampBasedKeyGenerator extends SimpleKeyGenerator {
enum TimestampType implements Serializable {
- UNIX_TIMESTAMP, DATE_STRING, MIXED, EPOCHMILLISECONDS
Review comment:
If this is set, we can make the timeunit be MILLISECONDS..
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
