hudi-bot opened a new issue, #15907:
URL: https://github.com/apache/hudi/issues/15907
I see that HoodieTableConfig has a config to let users to override timezone
for commit time generation. but looks like there are some places where we use
current system's zone instead of honoring the config.
{code:java}
public static Date parseDateFromInstantTime(String timestamp) throws
ParseException {
try {
// Enables backwards compatibility with non-millisecond granularity
instants
String timestampInMillis = timestamp;
if (isSecondGranularity(timestamp)) {
// Add milliseconds to the instant in order to parse successfully
timestampInMillis = timestamp + DEFAULT_MILLIS_EXT;
} else if (timestamp.length() > MILLIS_INSTANT_TIMESTAMP_FORMAT_LENGTH) {
// compaction and cleaning in metadata has special format. handling it
by trimming extra chars and treating it with ms granularity
timestampInMillis = timestamp.substring(0,
MILLIS_INSTANT_TIMESTAMP_FORMAT_LENGTH);
}
LocalDateTime dt = LocalDateTime.parse(timestampInMillis,
MILLIS_INSTANT_TIME_FORMATTER);
return Date.from(dt.atZone(ZoneId.systemDefault()).toInstant());
} catch (DateTimeParseException e) {
throw new ParseException(e.getMessage(), e.getErrorIndex());
}
} {code}
also, we are adding a guard rail to archival that archival should not go
past clean commits. when clean is based on hours, we have a fix which may not
work for other time zones. Ref patch: https://github.com/apache/hudi/pull/8422
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-6097
- Type: Improvement
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
