[
https://issues.apache.org/jira/browse/HUDI-6097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Guo updated HUDI-6097:
----------------------------
Component/s: archiving
table-service
> Follow up all timeline related apis and methods to honor timezone if
> overridden
> --------------------------------------------------------------------------------
>
> Key: HUDI-6097
> URL: https://issues.apache.org/jira/browse/HUDI-6097
> Project: Apache Hudi
> Issue Type: Improvement
> Components: archiving, reader-core, table-service, writer-core
> Reporter: sivabalan narayanan
> Priority: Major
>
> I see that HoodieTableConfig has a config to let users to override timezone
> for commit time generation. but looks like there are some places where we use
> current system's zone instead of honoring the config.
> {code:java}
> public static Date parseDateFromInstantTime(String timestamp) throws
> ParseException {
> try {
> // Enables backwards compatibility with non-millisecond granularity
> instants
> String timestampInMillis = timestamp;
> if (isSecondGranularity(timestamp)) {
> // Add milliseconds to the instant in order to parse successfully
> timestampInMillis = timestamp + DEFAULT_MILLIS_EXT;
> } else if (timestamp.length() > MILLIS_INSTANT_TIMESTAMP_FORMAT_LENGTH) {
> // compaction and cleaning in metadata has special format. handling it
> by trimming extra chars and treating it with ms granularity
> timestampInMillis = timestamp.substring(0,
> MILLIS_INSTANT_TIMESTAMP_FORMAT_LENGTH);
> }
> LocalDateTime dt = LocalDateTime.parse(timestampInMillis,
> MILLIS_INSTANT_TIME_FORMATTER);
> return Date.from(dt.atZone(ZoneId.systemDefault()).toInstant());
> } catch (DateTimeParseException e) {
> throw new ParseException(e.getMessage(), e.getErrorIndex());
> }
> } {code}
>
> also, we are adding a guard rail to archival that archival should not go past
> clean commits. when clean is based on hours, we have a fix which may not work
> for other time zones. Ref patch: https://github.com/apache/hudi/pull/8422
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
